Add the feature flag that allows users to opt out automatic UTF8 console encoding

[GangWang01]

Fixes #11850

Context

In non-English system especially like Japanese/Chinese, when redirecting the console output which is in UTF8 encoding to the pipe, the output is garbled. There is no way to detect the encoding of the destination. So add the feature flag that allows users to opt out automatic UTF8 console encoding.

Changes Made

Opt out automatic UTF8 console encoding by setting the environment variable CONSOLE_USE_DEFAULT_ENCODING to 1.

Testing

Manually tested.
For msbuild.exe

For dotnet build, it requires the change in SDK dotnet/sdk#51261.

Notes

[Copilot]

Pull Request Overview

This PR adds a feature flag to allow users to opt out of MSBuild's automatic UTF-8 console encoding. The change addresses garbled output issues in non-English systems (Japanese/Chinese) when redirecting console output to pipes.

• Introduces environment variable CONSOLE_USE_DEFAULT_ENCODING to control UTF-8 encoding behavior
• Wraps existing UTF-8 encoding logic in a conditional check

[baronfel]

It feels like we need an additional check here - if the CurrentUICulture is in a locale that doesn't support UTF8 then we should just not do automatic UTF8 encoding by default. The default behaviors/usage of the tool should just work for all users, regardless of locale.

[baronfel]

Any MSBuild-specific environment variable needs to start with an MSBUILD prefix.

[GangWang01]

It feels like we need an additional check here - if the CurrentUICulture is in a locale that doesn't support UTF8 then we should just not do automatic UTF8 encoding by default. The default behaviors/usage of the tool should just work for all users, regardless of locale.

@baronfel any reference of determining if the CurrentUICulture is in a locale that doesn't support UTF8? Currently, EncodingUtilities.CurrentPlatformIsWindowsAndOfficiallySupportsUTF8Encoding() is used to check if UTF8 is supported or not.

msbuild/src/MSBuild/XMake.cs

Lines 1938 to 1941 in 9373f72

	if (EncodingUtilities.CurrentPlatformIsWindowsAndOfficiallySupportsUTF8Encoding())
	#else
	if (EncodingUtilities.CurrentPlatformIsWindowsAndOfficiallySupportsUTF8Encoding()
	&& !CultureInfo.CurrentUICulture.TwoLetterISOLanguageName.Equals("en", StringComparison.InvariantCultureIgnoreCase))

.

For the issue, actually Windows with the locale ja-jp supports UTF8. MSbuild output encoded in UTF8 is displayed well. But after the output is piped with | more, it's garbled since the encoding of the pipe is Shift-JIS.