I take it you are using arcsoft?
I have never come across this, but I have never tried with a security camera.
2 tests I would do:
If you can, plug in a dvd player to the unit and see if it lets you record.
Try different software:
Debut (free and good)
http://www.nchsoftware.com/capture/When you go to the site, don't click on the links on the right hand side.
There is a download link for non commercial customers in the first paragraph
on the page.
Reboot the pc
Plug in EzCAP
Open Debut
Click on "device" at the top.