Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How can i merge multiple CSV files using scala? #267

Open
InduKrish opened this issue Feb 24, 2023 · 0 comments
Open

How can i merge multiple CSV files using scala? #267

InduKrish opened this issue Feb 24, 2023 · 0 comments
Labels

Comments

@InduKrish
Copy link

InduKrish commented Feb 24, 2023

I would like to merge these two csv files using scala,

R-1.csv
empNo, slotNo
13648,[49 50 51 52 53 54 55 56 41 42 43 44 45 46 47 48 1-3-5-7]
24824,[52 51 53 50 37 36 38]

R-2.csv
empNo, slotNo
23241,[48 50 51 52 53 54 55 56 41 42 43 44 45 46 47 48 1-4-5-7]
09344,[59 51 53 50 37 36 38]
I have these two files in scala, can someone please explain how i can merge these two files into one, and also looking for the way to merge these two files and change the order of records, something like this,

final csv :
empNo, slotNo
13648,[49 50 51 52 53 54 55 56 41 42 43 44 45 46 47 48 1-3-5-7]
09344,[59 51 53 50 37 36 38]
24824,[52 51 53 50 37 36 38]
23241,[48 50 51 52 53 54 55 56 41 42 43 44 45 46 47 48 1-4-5-7]

Can you please tell me how to remove csv header after merging and retain only one header and how to shuffle the csv rows after merging the file using scala?

can you please update the following code to retain only one header and shuffle the csv rows after merging? i want to use the merged csv file as a feeder for my gatling test.

  import java.io.File
  import com.github.tototoshi.csv._

  object CsvMergeExample extends App {
    // Read CSV files
    val reader1 = CSVReader.open(new File("R-1.csv"))
    val rows1 = reader1.all()
    reader1.close()

    val reader2 = CSVReader.open(new File("R-2.csv"))
    val rows2 = reader2.all()
    reader2.close()

    // Merge the two CSV files
    val mergedRows = rows1 ++ rows2

    // Sort the merged rows by the first column
    val sortedRows = mergedRows.sortBy(row => row.head.toInt)

    // Write the merged CSV to a new file
    val writer = CSVWriter.open(new File("merged.csv"))
    writer.writeAll(sortedRows)
    writer.close()
  }

i did have a stack overflow question on the same
https://stackoverflow.com/questions/75535626/how-can-i-merge-multiple-csv-files-using-scala
however the answer posted by someone is deleted now can you please assit?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

2 participants