[Bug] Import | JSON import fails - `Not valid JSON: expected value at line 1 column 1` - due to UTF8-BOM character

[mmailaender]

Getting Not valid JSON: expected value at line 1 column 1 while trying to import as json npx convex import --table accounts accounts.json

JSONs formats that I tried:

[
	{
		"accountNumber": "0001",
		"accountName": "reserviert"
	},
	{
		"accountNumber": "0040",
		"accountName": "reserviert"
	}
]

{
	"accounts": [
		{
			"accountNumber": "0001",
			"accountName": "reserviert"
		},
		{
			"accountNumber": "0040",
			"accountName": "reserviert"
		}
	]
}

[nipunn1313]

That first one should work. I had no issues.
Can you provide more information?

[nipunn1313]

you can also the jsonl format if you pass --format=jsonl (or rename the file to accounts.jsonl)

		{
			"accountNumber": "0001",
			"accountName": "reserviert"
		},
		{
			"accountNumber": "0040",
			"accountName": "reserviert"
		}

[mmailaender]

This is the file that I tried to upload that is throwing the error:
accounts.json

If I rename the file to jsonl and remove the surrounding array, I'm getting still this error Row 1 wasn't valid JSON: expected value at line 1 column 1.

Context: I've created the json file inside vscode on WSL.

[nipunn1313]

Aha. From the file you provided, there's an additional character before the initial [

xxd ~/Downloads/accounts.json | head
00000000: efbb bf5b 0d0a 097b 0d0a 0909 2261 6363  ...[...{...."acc
00000010: 6f75 6e74 4e75 6d62 6572 223a 2022 3030  ountNumber": "00
00000020: 3031 222c 0d0a 0909 2261 6363 6f75 6e74  01",...."account
00000030: 4e61 6d65 223a 2022 7265 7365 7276 6965  Name": "reservie
00000040: 7274 220d 0a09 7d2c 0d0a 097b 0d0a 0909  rt"...},...{....
00000050: 2261 6363 6f75 6e74 4e75 6d62 6572 223a  "accountNumber":
00000060: 2022 3030 3430 222c 0d0a 0909 2261 6363   "0040",...."acc
00000070: 6f75 6e74 4e61 6d65 223a 2022 7265 7365  ountName": "rese
00000080: 7276 6965 7274 220d 0a09 7d2c 0d0a 097b  rviert"...},...{
00000090: 0d0a 0909 2261 6363 6f75 6e74 4e75 6d62  ...."accountNumb

With some help from LLMs:

Looking at the xxd output, the first line shows:
00000000: efbb bf5b 0d0a 097b 0d0a 0909 2261 6363 ...[...{...."acc
The first three bytes are ef bb bf, which is the UTF-8 Byte Order Mark (BOM). This is a special sequence that indicates the file is encoded in UTF-8.
So before the initial [ character that starts your JSON array, there's a UTF-8 BOM. Many text editors and some JSON parsers handle this automatically, but it can sometimes cause issues with strict JSON parsers that don't expect it.

We can look into handling the BOM properly - but in the meantime try stripping the BOM from the start of the file (there's probably a VSCode setting or a tool that will do it).

If you (or anyone else out there) is feeling excited about a little rust development, you could try to implement it yourself (we'd take a PR).

[nipunn1313]

ended up "fixing" with a change to the CLI. It will go out soon.

b671cd8