About jpeg and future development of NFT marker creator

[kalwalt]

We continue here the discussion started on my PR for NFT kalwalt/jsartoolkit5#2 (comment)
@Carnaux we can leave here our thoughts and solutions for the problem.

[Carnaux]

To generate the marker files is needed some information about the image:

• Jpeg image
  • UInt8 Pixel Color Array
  • DPI
  • Width & Height
  • Number of Channels

The function "readImageFromFile" gets all the information to set a struct and then other functions can generate the marker files.

Using an HTML input gives only the Uint8 array, the width, and height. I tried setting a placeholder DPI, but it didn't work. I was trying to generate a new jpeg file and store it in the MEMFS and then read it "locally" with the GenTexData.c, so I wouldn't need to change the original code from the unity artoolkit.

Maybe the solution is easy, but I didn't have much time to work on it 😞

[kalwalt]

Thank you @Carnaux i will look into it when i have a bit of time.

[Carnaux]

@ThorstenBux Can you help me understand better the marker generation?

[ThorstenBux]

I can try :). What are your questions?

[Carnaux]

I solved most of my questions working today 😃, I'm generating the iset and fset files, but they don't work, they have drastically different file size than the generated by the genTexData.exe.

[nicolocarpignoli]

well, that's a great progress :) good luck, as soon as I will have time I will help you

[kalwalt]

I solved most of my questions working today , I'm generating the iset and fset files, but they don't work, they have drastically different file size than the generated by the genTexData.exe.

That really strange @Carnaux but as said Nicolo a great progress!

[Carnaux]

Just to document one of my attempts:

I tried to execute the genTexData.exe directly in node.js. If you don't pass a file as parameter the expected error message appears, so it runs and does "console.log", but when I pass the path for the file it won't show any sign that it is running.

I think that the .exe is waiting for a DPI input and is getting the code stuck. I tested with a simple hello world .exe compiled from c++ that just outputs and the code worked, when I added an input it got stuck. A possible solution is the use of child_process.spawn().

If the input problem gets solved, the data got by EXIF.js can create default values for the inputs of the .exe.

What the current code can do:

• Execute the genTexData.exe
• Get image via input( It is possible to get a temporary file and get its path, also possible to use the EXIF.js to get dpi, resolution, and color space)

[nicolocarpignoli]

@Carnaux I think that if you can come up with specific questions and maybe an online example to try (not mandatory) you can ask to Thorsten. He's very will to help. Just come up with some questions here and I will see to make it happen.

[Carnaux]

Update: I posted about an error dpi_num, but it was a typo error, and it was solved.

I think I figured out what was the deal. The way I was passing the pixel data as a parameter was wrong, solving that made the generation almost equal to the genTexData.exe, all logs are being shown, and all generation steps are seen to be working. But I noticed that the files still different sizes so they don't work with the NFT example, I believe it is the ARUint8 pixel array data structure, the feature extraction doesn't generate the correct values. @ThorstenBux, I saw in a forum that the ARUint8 pixel structure changes depending on the SO, it apply in this case too?

PS: Now all the steps of the marker generation work, but it takes 25 minutes to finish. The repository was updated, so anyone can test the generation times.

[kalwalt]

I will look at for the ARUint8 issue.
25 minutes It Is a huge time...

[Carnaux]

I developed a node app too, that uses the same markerCreator.min.js as the browser version(the current one), just to test generation times.

The results:

genTexData.exe: 3 min
Node app: 13 min
Browser version: 25 min

Some differences:

genTexData.exe:

Node app:

The values of filtered features are all slightly higher, but it doesn't log feature coordinates for high DPI values, only for 75 or below. Currently, it is faster than the browser, but I don't know how it will behave when it is hosted.

Browser:

The extracted and filtered values aren't correct, but it logs all coordinates for all DPIs.

[nicolocarpignoli]

Having wrong/few features could be the cause for the non-recognisation on Web.