The Rancho House is a house and music venue in the middle of IV, off a main road. It's traditional to keep the exact location a secret.
An article in The Bottom Line says "This serene sanctuary wins the title for oldest house in Isla Vista, built in 1885", but that's probably not true. If you know where it is, you can tell by comparing these aerial photos - it's not there in 1947, but it's there in 1956:
There are stories that perhaps the house was moved from its original location to IV, and that perhaps it was one of the houses from the old ranchos in the area. Rancho Dos Pueblos was a Mexican land grant given in 1842, which ran from present-day Goleta up the coast past Isla Vista (and you can trace land ownership from that grant to the present day). There's a lovely map here from 1871. It's likely that the remaining houses from that century are recognized as historic structures by the county (and they are probably adobe houses). For example, the Daniel A. Hill adobe at 35 South La Patera in Goleta, from about 1850, is recognized by the county as a "place of historic merit". The county's lists of places of historic merit and landmarks don't list any structures in Isla Vista. But who knows...