Be more careful about file reading

If SCons reads a file to interpret the contents, codecs are a concern.
The File node class has a get_text_contents() method which makes a best
effort at decoding bytes data, but there are other places that don't get
their file contents via that method, and so should do their own careful
decoding - but don't, they just read as text and hope it's okay.

Move the decode-bytes portion out of File.get_text_contents() to
SCons.Util.to_Text() so that everyone that needs this can call it.
Add a couple of additional known BOM codes (after consulting Python's
codecs module).

Note that while get_text_contents acts on nodes, the new (moved) routine
to_Text acts on passed bytes, so it can be used in a non-Node context
as well - for example the Java tool initializer reads a file and tries
to decode it, and can get it wrong (see #3569), this change provides it
some help.

Fixes #3569
FIxes #4462

Signed-off-by: Mats Wichmann <mats@linux.com>

0 files changed, 0 insertions, 0 deletions