East Melbourne is a small suburb attached directly to the CBD. A large proportion of this suburb is taken up by the Fitzroy Gardens.
In East Melbourne you will not find much in the way of cafes, shops or entertainment - unless you count Parliament House. It is possible to visit when Parliament is sitting -- see what Parliament actually do.
You can find some of the oldest buildings in Melbourne here.