Auto split a large .CSV-file in Mac OS X Terminal

How to split large CSV files in Mac OS X Terminal, the easy way. And add an extension to them also, so Mac OS X knows how to handle them.

How to split a large text file into parts of your choosing:

  • by line number;
  • or by file size (Kb or Mb);
  • and add a '.csv' extension to all splitted files

First example: auto split a single CSV file into multiple files of 200 lines each

Let's say I have one .csv-file that I want to split into multiple files that contain a maximum of 200 lines. I can use the following command:

split -l 200 split_me

Where split_me.csv is the filename of the CSV file you wish to split into chunks of 200 lines. The split function itself is smart enough to give the new files a name, so no need to worry about that.

For many cases this alone is enough. But if you want some additional options, read on.

Example 2: split into files of 1MB each, and add a '.csv' extension to all of them

Let's make it more interesting:

  • I want to split a file of 6.4MB into files of 1MB each,
  • and I want each resulting file to also have the .csv extension.

For that, we need to alter the split command a little bit.

First we used the -l flag, which separated the files by a number of lines. Now we're going to use the -b flag, which separates by size (bytes). Splitting it in files of 1MB is easy, just use 1m.

And additionally, we'll use a little loop and the mv-command to change the extensions.

First: split into chunks of 1mb each:

split -b 1m split_me.csv

And next, add the .csv extension to all of them:

for i in *; do mv "$i" "$i.csv"; done

Et voila! We can see the result:

Split Csv In Mac Terminal Results

Word of warning

You should know this, but just to make sure.

  1. Open the Terminal in the right folder.
  2. If you add the extensions with the command I provided, it will do this for ALL files in the folder. So make sure that files you do not want changed are outside of it
  3. You can see in the image that all new files are 1Mb each (except the last one of course), and they have all gained an extra suffix / extension of .csv

What's next?

If you think this was useful to you, it might be for others also. Feel free to share: