If the carpet was glued down you need to determine the type of adhesive that was used. Usually a dark or tan color adhesive is tar based and mineral spirits can be used.
Carpet Tape adhesive can be removed with an all-purpose adhesive remover. A yellow looking substance is generally carpet adhesive and can be lightly scraped or chipped off easily, and an all purpose adhesive remover will work.
Once the adhesive is removed, the floor can be sanded with a coarse paper and then re-sanded with finer paper.
In some cases, carpeting over a hardwood floor is installed with wooden tack strips. After the carpet has been removed and the tack strips are pulled up, wood putty then can be used to fill the holes. You can purchase colored wood putty to match the color of the stain if your floor is to remain uncovered. In this case you want to avoid using sandpaper, apply the colored wood putty using a 1" putty knife and take a rag with the appropriate thinner for the putty, fold it into a pad and wipe the excess from the surface while the putty is still wet.
WARNING!
Adhesive Removers are generally solvent base and are highly Flammable. Make sure all pilot lights and open flames are out, no smoking and LOTS of good ventilation. The fumes are very toxic and breathing them can be hazardous to your health. Before using any sanding or other electrical equipment allow the solvent to dry thourghly and ventilate the room for at least 24 hours. One spark and you won't have to worry about your floor any more.
Tip: We always damp mop the floor with water and a small amount of detergent prior to sanding so as to remove any residule solvent.