I have always used Sonic, a piece of software that came with my old laptop, to make SVCDs (Super Video CDs, which can be played on DVD players), but I just figured out a way to make SVCDs using free software on a Windows machine, so I thought I'd share the information here, since I didn't see any web page that put all of this information in one place.
You need 2 pieces of software -- eRightSoft Super to convert the video into VCD format, and CDR Tools Front End to take those video files and package them into a VCD.
After you install SUPER, select "SVCD Std. Compliant (MPG)" in the "1. Select the Output Container" section. Then drag and drop your movies into the area that says "DROP A VALID MULTIMEDIA FILE HERE". Then click on the "Encode (Active Files)" button. Now wait for it convert the videos into the proper format. (It could take a while.)
After that, run cdrtools Frontend and select "(S)VideoCD" from the tabs at the top of the screen, and drag and drop the videos you made in the earlier section into the main window. Then insert an empty CD into your CD writer and click "Start". Wait for it to finish.
Then you're done! You can now eject the SVCD and play it in a DVD player!