Arduino-CI is failing to find include file

[RobTillaart]

https://github.com/RobTillaart/MultiMap/runs/1637056493?check_suite_focus=true

MultiMap.h is a source file that contains one templated function, small enough to post here

#define MULTIMAP_LIB_VERSION "0.1.3"

#include "Arduino.h"

// note: the in array should have increasing values
template<typename T>
T multiMap(T val, T* _in, T* _out, uint8_t size)
{
    // take care the value is within range
    // val = constrain(val, _in[0], _in[size-1]);
    if (val <= _in[0]) return _out[0];
    if (val >= _in[size-1]) return _out[size-1];

    // search right interval
    uint8_t pos = 1;  // _in[0] allready tested
    while(val > _in[pos]) pos++;

    // this will handle all exact "points" in the _in array
    if (val == _in[pos]) return _out[pos];

    // interpolate in the right segment for the rest
    return (val - _in[pos-1]) * (_out[pos] - _out[pos-1]) / (_in[pos] - _in[pos-1]) + _out[pos-1];
}

The unit test is a simple test and that runs successful.

However compiling the examples , all complain that they cannot find the #include MultiMap.h
local compilation is no problem. So somehow the library is not findable.

Error message

Compiling multimap_NTC.ino for arduino:avr:uno... 
Last command:  $  /usr/local/bin/arduino-cli --format json compile --fqbn arduino:avr:uno --warnings all --dry-run /github/home/Arduino/libraries/MultiMap/examples/multimap_NTC/multimap_NTC.ino
/github/home/Arduino/libraries/MultiMap/examples/multimap_NTC/multimap_NTC.ino:10:10: fatal error: MultiMap.h: No such file or directory
 #include "MultiMap.h"
          ^~~~~~~~~~~~
compilation terminated.
Error during build: exit status 1

[RobTillaart]

Adding the MultiMap.h file in the same folder as the example results in success.
(tested with one example, but expected to work with all)

[per1234]

It seems to be caused by the incorrect filename case in the includes field of library.properties:
https://github.com/RobTillaart/MultiMap/blob/arduino-ci/library.properties#L10

includes=multimap.h

If I correct that to the right case;

includes=MultiMap.h

the error goes away.

I don't know enough about arduino_ci to understand why that error would would cause the CI run to fail in this way, but it's wrong regardless, so certainly worth fixing.

[ianfixes]

This seems like a good candate to test as-is with Arduino-CI/action@latest, I have some code to explicitly report on this. Can you give it shot?

[ianfixes]

I don't know enough about arduino_ci to understand why that error would would cause the CI run to fail in this way

If I'm reading that output properly, this error is coming from arduino-cli, running in a case-sensitive filesystem.

[per1234]

OK, I see it's this bug: arduino/arduino-cli#960
which was introduced in Arduino CLI 0.13.0 and has since been fixed (arduino/arduino-cli#987). So this would not occur if the latest release of Arduino CLI was used.

---

Even though the includes field value affecting compilation was a bug in Arduino CLI, having the wrong filename in that field is still a bug in the MultiMap library, since this will affect IDE users who use the "Sketch > Include Library > MultiMap" feature.

[ianfixes]

OK, I'm not completely clear on what's happening here nor what the proper behavior should be. Is the Arduino compiler and/or arduino-cli providing case-insensitive functionality even when operating in a case-sensitive filesytem (e.g. OSX, linux)?

[per1234]

OK, I'm not completely clear on what's happening here

If you're interested, see arduino/arduino-cli#960 and arduino/arduino-cli#987 for all the details.

Otherwise, just know that this is caused by a combination of a bug in the MultiMap library and Arduino CLI. There is nothing wrong with arduino_ci other than that it's using a version of Arduino CLI that's 3 months out of date.

So if you like, you can just say that it's RobTilaart's responsibility to fix the bug in MultiMap, or you can take this as the sign that you need to update to using Arduino CLI 0.14.0.

Is the Arduino compiler and/or arduino-cli providing case-insensitive functionality

No.

[ianfixes]

I did read that, but it didn't really talk about case sensitivity. Let me ask a different way: are these the proper fixes, given a specific arduino-cli version?

arduino-cli 0.13.0

  architectures=*
- includes=multimap.h
+ includes=MultiMap.h
  depends=

vs

arduino-cli 0.14.0

  architectures=*
- includes=multimap.h
  depends=

[ianfixes]

I don't meant to belabor this point, it's just that I've added code that tries to scan for incorrect data in the properties file:

d8a0a0b#diff-c8e39f6b37c608a31b7357c90dcd7b605242d3735e6bc87797cd32c90cd4f6c7R368

and from this conversation I can't tell if I've done it correctly.

[RobTillaart]

So if you like, you can just say that it's RobTilaart's responsibility to fix the bug in MultiMap

I'm on it

[RobTillaart]

The build was successful now.
imho a +1 for a syntax / content checker that verifies the lib.props file against the library itself.

[ianfixes]

+1 for a syntax / content checker that verifies the lib.props file against the library itself

That should be available in Arduino-CI/action@latest, did you try that out?

[RobTillaart]

+1 for a syntax / content checker that verifies the lib.props file against the library itself

That should be available in Arduino-CI/action@latest, did you try that out?

Not for this repo - I'll merge the succeeded arduino-ci build first and do a test afterwards

[RobTillaart]

Latest caught it

Run Arduino-CI/action@latest
/usr/bin/docker run --name cc4956f625cf8bc22b4d32a068ac733b9ef3c9_d43ebe --label cc4956 --workdir /github/workspace --rm -e HOME -e GITHUB_JOB -e GITHUB_REF -e GITHUB_SHA -e GITHUB_REPOSITORY -e GITHUB_REPOSITORY_OWNER -e GITHUB_RUN_ID -e GITHUB_RUN_NUMBER -e GITHUB_RETENTION_DAYS -e GITHUB_ACTOR -e GITHUB_WORKFLOW -e GITHUB_HEAD_REF -e GITHUB_BASE_REF -e GITHUB_EVENT_NAME -e GITHUB_SERVER_URL -e GITHUB_API_URL -e GITHUB_GRAPHQL_URL -e GITHUB_WORKSPACE -e GITHUB_ACTION -e GITHUB_EVENT_PATH -e GITHUB_ACTION_REPOSITORY -e GITHUB_ACTION_REF -e GITHUB_PATH -e GITHUB_ENV -e RUNNER_OS -e RUNNER_TOOL_CACHE -e RUNNER_TEMP -e RUNNER_WORKSPACE -e ACTIONS_RUNTIME_URL -e ACTIONS_RUNTIME_TOKEN -e ACTIONS_CACHE_URL -e GITHUB_ACTIONS=true -e CI=true -v "/var/run/docker.sock":"/var/run/docker.sock" -v "/home/runner/work/_temp/_github_home":"/github/home" -v "/home/runner/work/_temp/_github_workflow":"/github/workflow" -v "/home/runner/work/_temp/_runner_file_commands":"/github/file_commands" -v "/home/runner/work/MultiMap/MultiMap":"/github/workspace" cc4956:f625cf8bc22b4d32a068ac733b9ef3c9
Located arduino-cli binary...                         /usr/local/bin/arduino-cli
Environment variable CUSTOM_INIT_SCRIPT...                                    ''
Environment variable USE_SUBDIR...                                            ''
Installing library under test...                                               ✓
WARNING...     Installed library named 'MultiMap' has directory name 'workspace'
Library installed at...                  /github/home/Arduino/libraries/MultiMap
library.properties 'includes=' entry 'multimap.h' exists...                    ✗               <<<<<<<<<<<<<<<<<<<<<<
This may indicate a problem with your configuration; halting here
Failures: 1
Last message:  $  /usr/local/bin/arduino-cli --format json config dump
========== Stdout:
{
  "board_manager": {
    "additional_urls": []
  },
  "daemon": {
    "port": "50051"
  },
  "directories": {
    "data": "/github/home/.arduino15",
    "downloads": "/github/home/.arduino15/staging",
    "user": "/github/home/Arduino"
  },
  "logging": {
    "file": "",
    "format": "text",
    "level": "info"
  },
  "telemetry": {
    "addr": ":9090",
    "enabled": true
  }
}
========== Stderr:

The cross shows that it actually doesn't exist
The json dump does not give more information.

Better than the current master as it points to the right spot.

[ianfixes]

Thanks!

The JSON dump isn't necessarily helpful in all cases, it's just part of the failure reporter in the test runner script. I will make it more explicit that it's conveying the last interaction with the backend.

[per1234]

are these the proper fixes, given a specific arduino-cli version?
arduino-cli 0.13.0

 architectures=*
- includes=multimap.h
+ includes=MultiMap.h
 depends=

That's the proper fix regardless of Arduino CLI version. The problem is simply that the filename in the library.properties includes field is different from the filename in the library. In this case, that happened to be due to case mismatch, but the problem would have been the same if it was something like "foo.h".

The intended usage of the library.properties field is to customize which #include directives should be added to the sketch when someone selects the library from the Arduino IDE's Sketch > Include library menu. By default, this convenience feature will add #include directives to the sketch for every header file in the root of the library's source folder, but in some cases that's not the appropriate behavior, in which case the library author can specify the #include directives they want via the includes field. So with the previous library.properties configuration, if someone had selected Sketch > Include library > MultiMap, this line would have been added to the top of the currently opened sketch:

#include <multimap.h>

which would have resulted in a compilation error since the filename in the library is MultiMap.h. That's the bug in the MultiMap library.

But if the Arduino IDE's Sketch > Include library menu was not used then the incorrect includes field value should have no effect whatsoever on compilation. The fact that it did affect compilation is the bug in Arduino CLI 0.13.0.

[ianfixes]

...wow. if that's the case, the docs convey almost none of that.

includes - (available from Arduino IDE 1.6.10) (optional) a comma separated list of files to be added to the sketch as #include <...> lines. This property is used with the "Include library" command in the Arduino IDE. If the includes property is missing, all the header files (.h) on the root source folder are included.

Is there a way to submit improvements to that text? I'd suggest

includes - (available from Arduino IDE 1.6.10) (optional) a comma separated list, indicating the set of files that a sketch would need to include (as #include <...> lines) in order to use the library. This property informs the "Include library" command in the Arduino IDE, which auto-inserts those lines into a sketch. If the includes property is missing, all the header files (.h) on the root source folder are inserted.

Throughout this entire conversation I had been convinced that include= was a signal to the compiler or something.

That brings me to another question: is it allowed to include a file that doesn't end in .h? I may need to adjust logic if so.

[per1234]

Is there a way to submit improvements to that text?

Sure! The entire source of the documentation content of https://arduino.github.io/arduino-cli/dev/ is hosted in the arduino/arduino-cli repository:
https://github.com/arduino/arduino-cli/tree/master/docs

is it allowed to include a file that doesn't end in .h? I may need to adjust logic if so.

Yes. The header file extensions are merely a convention. There is no special treatment from the compiler based on which is used. The standard library actually doesn't use any extension, so you'll see things like this:

#include <functional>

However, the Arduino build system does require specific header file extensions in a couple of situations:

• Header files in the sketch folder. https://github.com/arduino/arduino-cli/blob/0.14.0/arduino/globals/globals.go#L29-L32
• The primary header files of a library: https://github.com/arduino/arduino-cli/blob/0.14.0/arduino/libraries/libraries.go#L221

[ianfixes]

Sorry... to clarify, is the IDE going to read the property include=functional and write #include <functional> in the sketch as part of the "Include library" menu command?

[per1234]

Yes. The Arduino IDE doesn't do any validation/restrictions on the contents of the includes field. It just splits on commas, trims whitespace and adds #include directives for each of the elements to the top of the sketch that is open at the time.

[ianfixes]

Opened arduino/arduino-cli#1124

Looks like I'll have to scale back the logic to just warnings about the include= entries not corresponding to files.

[per1234]

Opened arduino/arduino-cli#1124

Thanks! I'll do a review as soon as you run the formatting task and sign the CLA.