Skip to content

Load Unicode in the Metacello phase of the image build #18833

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: Pharo14
Choose a base branch
from
+37 −53
Open

Load Unicode in the Metacello phase of the image build #18833

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
9b472a4
Try to move Unicode loading later in the bootstrap
jecisc Nov 3, 2025
b846d35
Add logs
jecisc Nov 3, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 0 additions & 13 deletions bootstrap/scripts/01-initialization/02-initUnicode.st
View file
Open in desktop

This file was deleted.

6 changes: 1 addition & 5 deletions bootstrap/scripts/4-build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -138,7 +138,7 @@ ${VM} "${COMPILER_IMAGE_NAME}.image" # I have to run once the image so the next

echo $(date -u) "[Compiler] Adding more Kernel packages"
${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" perform --save BasicHermesTool load: --as-array Clap-Core.hermes Clap-Commands-Pharo.hermes Hermes-Extensions.hermes
${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" loadHermes Math-Operations-Extensions.hermes Debugging-Core.hermes System-NumberPrinting.hermes System-Time.hermes Multilingual-Encodings.hermes ReflectionMirrors-Primitives.hermes NumberParser.hermes --save --no-fail-on-undeclared
${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" loadHermes Math-Operations-Extensions.hermes Debugging-Core.hermes System-NumberPrinting.hermes System-Time.hermes ReflectionMirrors-Primitives.hermes NumberParser.hermes --save --no-fail-on-undeclared

# Now that System-Time is loaded, we can initialize the version
${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" perform --save SystemVersion setMajor:minor:patch:suffix:build:commitHash: ${PHARO_MAJOR} ${PHARO_MINOR} ${PHARO_PATCH} ${PHARO_SUFFIX} ${BUILD_NUMBER} ${PHARO_COMMIT_HASH}
Expand All @@ -154,10 +154,6 @@ echo $(date -u) "[Compiler] Installing compiler through Hermes"
${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" loadHermes UIManager.hermes Debugging-Utils.hermes OpalCompiler-Core.hermes Deprecation.hermes DebugInfo.hermes CodeImport.hermes CodeImport-Commands.hermes --save --no-fail-on-undeclared
${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" eval --save "SystemEnvironment deprecatedAliases: { #SystemDictionary }." # This line should be removed in Pharo 14 since it is for backward compatibility.
${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" st ${BOOTSTRAP_REPOSITORY}/bootstrap/scripts/01-initialization/01-init.st --no-source --save --quit

echo $(date -u) "[Compiler] Initializing Unicode"
${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" st ${BOOTSTRAP_REPOSITORY}/bootstrap/scripts/01-initialization/02-initUnicode.st --no-source --save --quit "${BOOTSTRAP_REPOSITORY}/resources/unicode/"

${VM} "${COMPILER_IMAGE_NAME}.image" "${IMAGE_FLAGS}" loadHermes Random-Core.hermes System-Hashing.hermes Network-UUID.hermes FileSystem-Path.hermes FileSystem-Core.hermes FileSystem-Disk.hermes --save --no-fail-on-undeclared
zip "${COMPILER_IMAGE_NAME}.zip" "${COMPILER_IMAGE_NAME}.image"

Expand Down
2 changes: 1 addition & 1 deletion bootstrap/scripts/generateKernelHermesFiles.st
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ repository := TonelRepository new

Transcript nextPutAll: '[Hermes] - Generating Environment from repository'; cr.

toExport := #BaselineOfPharoBootstrap asClass compilerPackageNames, {'Hermes-Extensions'}, #BaselineOfPharoBootstrap asClass multilingualPackageNames, #BaselineOfPharoBootstrap asClass kernelAdditionalPackagesNames.
toExport := #BaselineOfPharoBootstrap asClass compilerPackageNames, {'Hermes-Extensions'}, #BaselineOfPharoBootstrap asClass kernelAdditionalPackagesNames.

packageNames := #BaselineOfPharoBootstrap asClass kernelPackageNames, toExport.
environment := repository asRing2EnvironmentWith: packageNames.
Expand Down
12 changes: 0 additions & 12 deletions src/BaselineOfPharoBootstrap/BaselineOfPharoBootstrap.class.st
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,12 +36,6 @@ BaselineOfPharoBootstrap class >> kernelPackageNames [
^ self packagesOfGroupNamed: #KernelGroup
]

{ #category : 'accessing' }
BaselineOfPharoBootstrap class >> multilingualPackageNames [

^ self packagesOfGroupNamed: #MultilingualGroup
]

{ #category : 'traversing' }
BaselineOfPharoBootstrap class >> packagesRecursiveIn: aGroupName [

Expand Down Expand Up @@ -99,8 +93,6 @@ BaselineOfPharoBootstrap >> baseline: spec [

spec package: 'Math-Operations-Extensions'.

spec package: 'Multilingual-Encodings'.

spec package: 'Network-UUID'.

spec package: 'Transcript-NonInteractive'.
Expand Down Expand Up @@ -192,9 +184,6 @@ BaselineOfPharoBootstrap >> baseline: spec [
'System-NumberPrinting'.
}.

spec group: 'MultilingualGroup' with: {
'Multilingual-Encodings' }.

spec group: 'CompilerGroup' with: {
'UIManager'.
'AST-Core'.
Expand All @@ -218,7 +207,6 @@ BaselineOfPharoBootstrap >> baseline: spec [

spec group: 'ToLoadByHermes' with: {
'AdditionalPackages'.
'MultilingualGroup'.
'CompilerGroup'.
'FileSystemGroup'.
'Hermes-Extensions'.
Expand Down
22 changes: 11 additions & 11 deletions src/BaselineOfUnifiedFFI/BaselineOfUnifiedFFI.class.st
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,17 +7,17 @@ Class {

{ #category : 'baseline' }
BaselineOfUnifiedFFI >> baseline: spec [

<baseline>

spec for: #common do: [
spec
"unified ffi"
package: 'UnifiedFFI';
package: 'UnifiedFFI-Tests' with: [ spec requires: #('UnifiedFFI') ];
package: 'System-OSEnvironments' with: [ spec requires: #('UnifiedFFI') ].
spec
group: 'default' with: #('core');
group: 'minimal' with: #('UnifiedFFI' 'System-OSEnvironments');
group: 'core' with: #('minimal' 'UnifiedFFI-Tests') ]
spec
package: 'UnifiedFFI';
package: 'UnifiedFFI-Tests' with: [ spec requires: #( 'UnifiedFFI' ) ];
package: 'System-OSEnvironments' with: [ spec requires: #( 'UnifiedFFI' ) ];
package: 'Multilingual-Encodings' with: [ spec requires: #( 'System-OSEnvironments' ) ]. "This package should probably not be here, but it depends on System-OSEnvironments "

spec
group: 'default' with: #( 'core' );
group: 'minimal' with: #( 'UnifiedFFI' 'System-OSEnvironments' 'Multilingual-Encodings');
group: 'core' with: #( 'minimal' 'UnifiedFFI-Tests' ) ]
]
5 changes: 0 additions & 5 deletions src/Kernel/ManifestKernel.class.st
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -39,8 +39,3 @@ ManifestKernel class >> ruleRBBadMessageRuleV1FalsePositive [
ManifestKernel class >> ruleRBRefersToClassRuleV1FalsePositive [
^ #(#(#(#RGMethodDefinition #(#'Character class' #allByteCharacters #true)) #'2019-01-28T20:55:28.683846+01:00') )
]

{ #category : 'code-critics' }
ManifestKernel class >> ruleReDeadMethodV1FalsePositive [
^ #(#(#(#RGMethodDefinition #(#'Character class' #characterSet: #true)) #'2019-01-28T20:56:23.286894+01:00') )
]
23 changes: 21 additions & 2 deletions src/Multilingual-Encodings/Unicode.class.st
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -472,6 +472,7 @@ Unicode class >> initialize [
2) unicode data
3) case mappings
"

"If you are in a full image, you can try the following Zinc scripts to download the file automatically:

UIManager default informUserDuring: [ :bar| | result |
Expand All @@ -489,8 +490,26 @@ Unicode class >> initialize [
self parseCaseMappingFrom: result contents readStream ].
"

EncodedCharSet declareEncodedCharSet: self atIndex: 0+1.
EncodedCharSet declareEncodedCharSet: self atIndex: 256
EncodedCharSet declareEncodedCharSet: self atIndex: 0 + 1.
EncodedCharSet declareEncodedCharSet: self atIndex: 256.

self initializeTagConstants.

Smalltalk os environment
at: #BOOTSTRAP_REPOSITORY
ifPresent: [ :path |
| dataFilePath caseFolding unicodeData |
dataFilePath := path , '/resources/unicode/'.
('Initializing Unicode with path ', dataFilePath) traceCr.
unicodeData := dataFilePath , 'UnicodeData.txt'.
caseFolding := dataFilePath , 'CaseFolding-8.0.0.txt'.

self
parseUnicodeDataFrom: (ZnCharacterReadStream on: (File named: unicodeData) readStream encoding: 'utf8');
parseCaseMappingFrom: (ZnCharacterReadStream on: (File named: caseFolding) readStream encoding: 'utf8').

Character characterSet: self ]
ifAbsent: [ Warning signal: 'Cannot initialize Unicode outside the Pharo bootstrap' ]
]

{ #category : 'class initialization' }
Expand Down
7 changes: 3 additions & 4 deletions src/System-Dependencies-Tests/SystemDependenciesTest.class.st
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ SystemDependenciesTest >> externalDependendiesOf: packagesCollection [
{ #category : 'packages' }
SystemDependenciesTest >> kernelLayerPackages [

^ (BaselineOfPharoBootstrap kernelPackageNames , BaselineOfPharoBootstrap multilingualPackageNames , BaselineOfPharoBootstrap kernelAdditionalPackagesNames)
^ (BaselineOfPharoBootstrap kernelPackageNames , BaselineOfPharoBootstrap kernelAdditionalPackagesNames)
asSet
]

Expand Down Expand Up @@ -111,7 +111,6 @@ SystemDependenciesTest >> metacelloPackageNames [
^ BaselineOfPharoBootstrap kernelPackageNames,
BaselineOfPharoBootstrap compilerPackageNames,
BaselineOfPharoBootstrap fileSystemPackageNames,
BaselineOfPharoBootstrap multilingualPackageNames,
BaselineOfPharoBootstrap kernelAdditionalPackagesNames,
BaselineOfMonticello corePackageNames,
BaselineOfMonticello remoteRepositoriesPackageNames,
Expand Down Expand Up @@ -224,7 +223,7 @@ SystemDependenciesTest >> testExternalFileSystemDependencies [

| dependencies |
dependencies := self externalDependendiesOf:
BaselineOfPharoBootstrap kernelPackageNames , BaselineOfPharoBootstrap multilingualPackageNames
BaselineOfPharoBootstrap kernelPackageNames
, BaselineOfPharoBootstrap kernelAdditionalPackagesNames , BaselineOfPharoBootstrap compilerPackageNames
, BaselineOfPharoBootstrap fileSystemPackageNames.

Expand All @@ -244,7 +243,7 @@ If you will break or weaken this test, a puppy will die!!!
| dependencies |

dependencies := self
externalDependendiesOf: BaselineOfPharoBootstrap kernelPackageNames , BaselineOfPharoBootstrap multilingualPackageNames, BaselineOfPharoBootstrap kernelAdditionalPackagesNames.
externalDependendiesOf: BaselineOfPharoBootstrap kernelPackageNames, BaselineOfPharoBootstrap kernelAdditionalPackagesNames.

self assertCollection: dependencies hasSameElements: self knownKernelDependencies
]
Expand Down