Clarification of the scripts caching vs reuse of context

[and-za]

Hi. I probably does not understand things around scripts caching well enough. Here is my "problem".

I have a very simple JavaScript script which just concatenates two strings. When I call this script in a loop of lets say 100 000 iterations, and in each iteration I use the same Context, then it's pretty quick.

But when I want to use "engine sharing" instead, that means in each iteration create new Context but with shared Engine, then it is very slow.

My understanding was, that the script evaluated in first (few) iterations would be parsed, optimized and cached in the engine, so each subsequent context would benefit from the optimizations/parsing in the first (few) iterations. But that does not seem to be true. Could you please clarify to me, how is it intended to work and why there is so big difference?

I have tested it with OpenJDK 21, but also with Graal VM 21/24. The behavior is more or less the same. Graal version 23.1.2, but also the 24.2.1 on JDK 24.

Here is the code which I used to "benchmark" it:

public class GraalJsExample {

    private static final Source SCRIPT = Source.create("js", "param1 + param2");

    public static void main(String[] args) {
        System.out.println("Generating random strings...");
        final var strings = createRandomStrings(100_001);

        System.out.println("Benchmarking");
        for (int i = 0; i < 10; i++) {
            System.out.println("Iteration " + (i + 1) + " ---------------------");
            final long startLoop = System.nanoTime();
            // Uncomment one of bellow two lines
            benchmarkWithContextReuse(strings);
            //benchmarkWithSharedEngine(strings);
            final long endLoop = System.nanoTime();

            final double totalSeconds = (endLoop - startLoop) / 1_000_000_000.0;
            System.out.printf("JS execution loop completed in: %.3f seconds%n", totalSeconds);
            System.out.printf("Average time per iteration: %.3f ms%n",
                    (endLoop - startLoop) / 1_000_000.0 / (strings.size() -1));
        }
    }

    private static void benchmarkWithContextReuse(List<String> strings) {
        int charactersCount = 0;
        try (final Context context = Context.newBuilder()
                .allowHostAccess(HostAccess.UNTRUSTED)
                .build()) {
            for (int i = 1; i < strings.size(); i++) {
                charactersCount += concatWithJs(context, strings.get(i - 1), strings.get(i)).length();
            }

        }
        System.out.printf("Number of characters (Just to use the script result for something): %d.%n", charactersCount);
    }

    private static void benchmarkWithSharedEngine(List<String> strings) {
        int charactersCount = 0;
        try (final Engine engine = Engine.create()) {
            for (int i = 1; i < strings.size(); i++) {
                try (final Context context = Context.newBuilder()
                        .engine(engine)
                        .allowHostAccess(HostAccess.UNTRUSTED)
                        .build()) {

                    charactersCount += concatWithJs(context, strings.get(i - 1), strings.get(i)).length();
                }
            }
        }

        System.out.printf("Number of characters (Just to use the script result for something): %d.%n", charactersCount);
    }

    private static String concatWithJs(Context context, String param1, String param2) {
            context.getBindings("js").putMember("param1", param1);
            context.getBindings("js").putMember("param2", param2);
            return context.eval(GraalJsExample.SCRIPT).asString();
    }

    private static String concatWithGroovy(Script script, String param1, String param2) {
        final Binding binding = new Binding(Map.of("param1", param1, "param2", param2));
        script.setBinding(binding);
        return script.run().toString();
    }

    private static List<String> createRandomStrings(int count) {
        final Random random = new Random();
        final List<String> result = new ArrayList<>(count);

        // Generate random strings of length 5-15 characters
        for (int i = 0; i < count; i++) {
            final int length = random.nextInt(11) + 5; // Random length between 5 and 15
            final StringBuilder sb = new StringBuilder(length);

            for (int j = 0; j < length; j++) {
                // Generate random printable ASCII characters (from '!' to '~')
                final char c = (char) (random.nextInt(94) + 33);
                sb.append(c);
            }

            result.add(sb.toString());
        }

        return result;
    }
}

[woess]

Given that your js code snippet is trivial, I suspect that you're measuring mostly Context initialization overhead and thus not seeing the benefits of code caching. Context creation isn't free, but usually dominated by the JS code.
If your JS code is very short-running and you don't need isolated contexts, I would suggest to consider reusing thread-local contexts.

[chumer]

If you can't reuse the context, as the global state of the scripts should not be shared, then another option is to share an engine across contexts. There are some docs here:
https://www.graalvm.org/latest/reference-manual/embed-languages/#code-caching-across-multiple-contexts