Public Records Office Victoria hold a number of records relating to the Victorian State Government. Think about how your ancestor interacted with the government and which department that may have created a record. Records date from 1836.
The researcher landing page is a good introduction.
Free online indexes include passenger lists, wills and probate, education and more.