Skip to content

issue: replicating issue 1098 #1101

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 1 commit into
base: master
Choose a base branch
from
Draft

issue: replicating issue 1098 #1101

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
150 changes: 150 additions & 0 deletions replications/issue-1098.js
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
/**
* Bug Replication for Issue #1098: transformHeader renaming and ignoring not working as expected
* https://github.com/mholt/PapaParse/issues/1098
*
* This script demonstrates the bug where using transformHeader to both rename headers and ignore
* columns by returning null causes all headers to be renamed to null.
*
* FINDINGS:
* - The bug is more severe than originally reported - it causes a TypeError crash
* - The crash occurs because stripBom() in papaparse.js tries to call charCodeAt() on null
* - The transformHeader function is called multiple times on the same headers
* - Even the workaround of returning empty string doesn't work properly
* - The issue affects both object-based header mapping and array-based header filtering
*/

const Papa = require('../papaparse.js');

// Sample CSV data with multiple columns
const csvData = `Code Set,Code Name,Code Value,description,extra_column1,extra_column2
CS001,Test Code,TC123,This is a test code,Extra Data 1,Extra Data 2
CS002,Another Code,AC456,Another test code,More Extra,Even More Extra
CS003,Third Code,TC789,Third test code,Additional,Additional Data`;

console.log('Issue #1098 Replication Script');
console.log('==============================\n');

// Header mapping as described in the issue
const headerMap = {
'Code Set': 'codeSet',
'Code Name': 'codeName',
'Code Value': 'codeValue',
};

// Test Case 1: Working case - renaming headers but keeping unmapped ones
console.log('Test Case 1: Renaming headers (keeping unmapped headers) - WORKING');
console.log('-------------------------------------------------------------------');
const config1 = {
header: true,
skipEmptyLines: true,
transformHeader: (header) => {
if (header in headerMap) {
console.log(` Transforming "${header}" to "${headerMap[header]}"`);
return headerMap[header];
}
console.log(` Keeping header "${header}" as is`);
return header;
}
};

const result1 = Papa.parse(csvData, config1);
console.log('\nParsed data (first row):');
console.log(JSON.stringify(result1.data[0], null, 2));
console.log(`\nTotal rows parsed: ${result1.data.length}`);
console.log('\n');

// Test Case 2: Broken case - trying to ignore unmapped headers by returning null
console.log('Test Case 2: Renaming headers and ignoring unmapped (returning null) - BROKEN');
console.log('------------------------------------------------------------------------------');
const config2 = {
header: true,
skipEmptyLines: true,
transformHeader: (header) => {
if (header in headerMap) {
console.log(` Transforming "${header}" to "${headerMap[header]}"`);
return headerMap[header];
}
console.log(` Returning null for header "${header}"`);
return null;
}
};

try {
const result2 = Papa.parse(csvData, config2);
console.log('\nParsed data (first row):');
console.log(JSON.stringify(result2.data[0], null, 2));
console.log(`\nTotal rows parsed: ${result2.data.length}`);
} catch (error) {
console.log('\nERROR: ' + error.message);
console.log('Stack trace:', error.stack.split('\n').slice(0, 5).join('\n'));
}
console.log('\n');

// Test Case 3: Using array of headers without renaming (mentioned as working in the issue)
console.log('Test Case 3: Using array of headers without renaming - TESTING');
console.log('---------------------------------------------------------------');
const validHeaders = ['Code Set', 'Code Name', 'Code Value'];
const config3 = {
header: true,
skipEmptyLines: true,
transformHeader: (header) => {
if (validHeaders.includes(header)) {
console.log(` Keeping header "${header}"`);
return header;
}
console.log(` Returning null for header "${header}"`);
return null;
}
};

try {
const result3 = Papa.parse(csvData, config3);
console.log('\nParsed data (first row):');
console.log(JSON.stringify(result3.data[0], null, 2));
console.log(`\nTotal rows parsed: ${result3.data.length}`);
} catch (error) {
console.log('\nERROR: ' + error.message);
console.log('Stack trace:', error.stack.split('\n').slice(0, 5).join('\n'));
}
console.log('\n');

// Test Case 4: Potential workaround - returning empty string instead of null
console.log('Test Case 4: Workaround - returning empty string instead of null');
console.log('-----------------------------------------------------------------');
const config4 = {
header: true,
skipEmptyLines: true,
transformHeader: (header) => {
if (header in headerMap) {
console.log(` Transforming "${header}" to "${headerMap[header]}"`);
return headerMap[header];
}
console.log(` Returning empty string for header "${header}"`);
return ''; // Return empty string instead of null
}
};

try {
const result4 = Papa.parse(csvData, config4);
console.log('\nParsed data (first row):');
console.log(JSON.stringify(result4.data[0], null, 2));
console.log(`\nTotal rows parsed: ${result4.data.length}`);
} catch (error) {
console.log('\nERROR: ' + error.message);
console.log('Stack trace:', error.stack.split('\n').slice(0, 5).join('\n'));
}
console.log('\n');

// Summary
console.log('SUMMARY');
console.log('=======');
console.log('\nExpected behavior:');
console.log('- Test Case 1: Headers should be renamed according to headerMap, unmapped headers kept as is');
console.log('- Test Case 2: Headers should be renamed according to headerMap, unmapped headers should be ignored/removed');
console.log('- Test Case 3: Headers in validHeaders array should be kept, others should be ignored/removed');
console.log('- Test Case 4: Workaround test - using empty string instead of null');
console.log('\nActual behavior:');
console.log('- Test Case 1: Works as expected');
console.log('- Test Case 2: BUG - Causes TypeError when trying to process null headers');
console.log('- Test Case 3: Same issue - TypeError when returning null');
console.log('- Test Case 4: Check if empty string works as a workaround');
108 changes: 108 additions & 0 deletions replications/issue-1098.log
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,108 @@
Issue #1098 Replication Script
==============================

Test Case 1: Renaming headers (keeping unmapped headers) - WORKING
-------------------------------------------------------------------
Transforming "Code Set" to "codeSet"
Transforming "Code Name" to "codeName"
Transforming "Code Value" to "codeValue"
Keeping header "description" as is
Keeping header "extra_column1" as is
Keeping header "extra_column2" as is
Keeping header "codeSet" as is
Keeping header "codeName" as is
Keeping header "codeValue" as is
Keeping header "description" as is
Keeping header "extra_column1" as is
Keeping header "extra_column2" as is

Parsed data (first row):
{
"codeSet": "CS001",
"codeName": "Test Code",
"codeValue": "TC123",
"description": "This is a test code",
"extra_column1": "Extra Data 1",
"extra_column2": "Extra Data 2"
}

Total rows parsed: 3


Test Case 2: Renaming headers and ignoring unmapped (returning null) - BROKEN
------------------------------------------------------------------------------
Transforming "Code Set" to "codeSet"
Transforming "Code Name" to "codeName"
Transforming "Code Value" to "codeValue"
Returning null for header "description"
Returning null for header "extra_column1"
Returning null for header "extra_column2"
Returning null for header "codeSet"
Returning null for header "codeName"
Returning null for header "codeValue"

ERROR: Cannot read properties of null (reading 'charCodeAt')
Stack trace: TypeError: Cannot read properties of null (reading 'charCodeAt')
at stripBom (/Users/dboskovic/Code/Contrib/PapaParse/papaparse.js:190:14)
at addHeader (/Users/dboskovic/Code/Contrib/PapaParse/papaparse.js:1234:14)
at Array.forEach (<anonymous>)
at fillHeaderFields (/Users/dboskovic/Code/Contrib/PapaParse/papaparse.js:1244:23)


Test Case 3: Using array of headers without renaming - TESTING
---------------------------------------------------------------
Keeping header "Code Set"
Keeping header "Code Name"
Keeping header "Code Value"
Returning null for header "description"
Returning null for header "extra_column1"
Returning null for header "extra_column2"
Keeping header "Code Set"
Keeping header "Code Name"
Keeping header "Code Value"

ERROR: Cannot read properties of null (reading 'charCodeAt')
Stack trace: TypeError: Cannot read properties of null (reading 'charCodeAt')
at stripBom (/Users/dboskovic/Code/Contrib/PapaParse/papaparse.js:190:14)
at addHeader (/Users/dboskovic/Code/Contrib/PapaParse/papaparse.js:1234:14)
at Array.forEach (<anonymous>)
at fillHeaderFields (/Users/dboskovic/Code/Contrib/PapaParse/papaparse.js:1244:23)


Test Case 4: Workaround - returning empty string instead of null
-----------------------------------------------------------------
Transforming "Code Set" to "codeSet"
Transforming "Code Name" to "codeName"
Transforming "Code Value" to "codeValue"
Returning empty string for header "description"
Returning empty string for header "extra_column1"
Returning empty string for header "extra_column2"
Returning empty string for header "codeSet"
Returning empty string for header "codeName"
Returning empty string for header "codeValue"
Returning empty string for header ""
Returning empty string for header "_1"
Returning empty string for header "_2"

Parsed data (first row):
{
"": "Extra Data 2"
}

Total rows parsed: 3


SUMMARY
=======

Expected behavior:
- Test Case 1: Headers should be renamed according to headerMap, unmapped headers kept as is
- Test Case 2: Headers should be renamed according to headerMap, unmapped headers should be ignored/removed
- Test Case 3: Headers in validHeaders array should be kept, others should be ignored/removed
- Test Case 4: Workaround test - using empty string instead of null

Actual behavior:
- Test Case 1: Works as expected
- Test Case 2: BUG - Causes TypeError when trying to process null headers
- Test Case 3: Same issue - TypeError when returning null
- Test Case 4: Check if empty string works as a workaround