Biggest factor is your age. Profile info says you're 24, so that's part of it. When you're 34, you'll see a marked improvement.
Next biggest factor is driving history. If you've had accidents or tickets it will drive up the cost.
Next, as others have mentioned is the type of bike. Sport bikes can be like 5x the cost of cruisers with the same size engine, depending upon the state you live in. My Triumph cruiser was $175 a year. My Ducati Superbike is $500 a year. Some companies wanted over $700 for the same coverage.
Next is the state in which you live. In Virginia, my insurance was easily double what it was for the same bike in Texas
Next is the age of the bike. new bikes cost more to insure than ones, say, 5 or more years old.
Also take into account your zip code. Areas with higher incidents of accident and theft can drive up your cost, even though it has nothing to do with you personally.
It can vary from company to company, so shop around. Progressive, Geico and Allstate usually have competitive rates. Farmers, State Farm and Nationwide are usually high as fuck, at least in my experience. If you can find an independent insurance broker, they can shop multiple companies for you including some lesser known ones. It's been a while since I was young but I recall Dairyland being cheap back then.