Getting back into cycling

It has been over a decade since I got on a bicycle. Hard to believe but I don't feel alone when speaking with other people about this topic. I only know one person who bikes to work though they live very close to the office.
I'm not sure where to start because I don't want to spend alot of money and quit using it. I also do not want to buy a cheap bike from a big department store that makes me not want to continue cycling. Stolen bikes are a big thing so buying used will be a gamble. I'm looking forward to a new hobby and the bonus of exercise I will get from cycling. I need to think of my health and this will be a good start.
For now, I will research local bike shops and get to know the people that work there. I'm sure they can provide advice and point me in the right direction.