We really shouldn't override what the user has set externally. While it's true
that `-Wall` is set in `DEBUG_CFLAGS` later, dropping it from `CFLAGS` breaks
Perl detection somehow. Instead of going through the work to figure out how to
fix that, just stop modifying `CFLAGS` from the beginning.
Testing Done:
Compiled on Rawhide with the attached patch.