Do you get any noise? What generation of iMac? Did you do anything else when you replace the HD? RAM perhaps?
Could be the PRAM. Try booting while holding the 'P' and 'R' down. Do this for about 10 seconds and you should start to hear a fan run up. It will then shut down and do this again. Hold the keys through three cycles then release them during the fourth boot and it should get you to the gray Apple screen.
If the PRAM doesn't fix it, it could be loose RAM. Verify that they are seated properly (though you would usually get three beeps if this was the case). There is also a possibility, depending on the generation, that you might have bumped one of the tiny little connectors on the LogicBoard near the HD. Try pulling the shell apart again and making sure all the connectors are properly connected.
Could also need the LB reset. There is a little button on the LB that you hold down for a few seconds and that will rest the LB. Its in a different spot on every generation. Also might be the LB battery, little watch battery that helps it remember time, ect. Might need replacing.
