One thing I had forgotten was that pdflatex prefers files with Unix end-of-line[1] and was throwing this error:
! Undefined control sequence.
l.30 }\SkipToFmtEnd
? q
OK, entering \batchmode
A solution is to run dos2unix on the generated .tex file after running lhs2TeX:
$ lhs2TeX -o HelloWorld.tex HelloWorld.lhs
$ dos2unix HelloWorld.tex
$ pdflatex HelloWorld.tex
[1] Generally I use LaTeX and dvips, pvipdfm on Windows which seem agnostic about linefeed format.