{"id":2217,"date":"2019-09-13T11:32:43","date_gmt":"2019-09-13T09:32:43","guid":{"rendered":"http:\/\/jakisproblem.pl\/?p=2217"},"modified":"2019-09-13T11:32:43","modified_gmt":"2019-09-13T09:32:43","slug":"how-to-combine-multiple-csv-files-with-8-lines-of-code","status":"publish","type":"post","link":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/2019\/09\/13\/how-to-combine-multiple-csv-files-with-8-lines-of-code\/","title":{"rendered":"How to combine multiple CSV files with 8 lines of code"},"content":{"rendered":"\n<h4 class=\"wp-block-heading\" id=\"step-1-import-packages-and-set-the-working-directory\">Step 1: Import packages and set the working directory<\/h4>\n\n\n\n<p>Change \u201c\/mydir\u201d to your desired working directory.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>import os\nimport glob\nimport pandas as pd\nos.chdir(\"\/mydir\")<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"step-2-use-glob-to-match-the-pattern-csv-\">Step 2: Use glob to match the pattern \u2018csv\u2019<\/h4>\n\n\n\n<p>Match the pattern (\u2018csv\u2019) and save the list of file names in the \u2018all_filenames\u2019 variable. You can check out&nbsp;<a href=\"https:\/\/regexr.com\/\">this link<\/a>&nbsp;to learn more about regular expression matching.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>extension = 'csv'\nall_filenames = [i for i in glob.glob('*.{}'.format(extension))]<\/code><\/pre>\n\n\n\n<h4 class=\"wp-block-heading\" id=\"step-3-combine-all-files-in-the-list-and-export-as-csv\">Step 3: Combine all files in the list and export as CSV<\/h4>\n\n\n\n<p>Use pandas to concatenate all files in the list and export as CSV. The output file is named \u201ccombined_csv.csv\u201d located in your working directory.<\/p>\n\n\n\n<pre class=\"wp-block-code\"><code>#combine all files in the list\ncombined_csv = pd.concat([pd.read_csv(f) for f in all_filenames ])\n#export to csv\ncombined_csv.to_csv( \"combined_csv.csv\", index=False, encoding='utf-8-sig')<\/code><\/pre>\n\n\n\n<p>encoding = \u2018utf-8-sig\u2019 is added to overcome the issue when exporting \u2018Non-English\u2019 languages.<\/p>\n\n\n\n<p>And\u2026it\u2019s done!<\/p>\n\n\n\n<p><a href=\"https:\/\/www.freecodecamp.org\/news\/how-to-combine-multiple-csv-files-with-8-lines-of-code-265183e0854\/\">https:\/\/www.freecodecamp.org\/news\/how-to-combine-multiple-csv-files-with-8-lines-of-code-265183e0854\/<\/a><\/p>\n","protected":false},"excerpt":{"rendered":"<p class=\"excerpt\">Step 1: Import packages and set the working directory Change \u201c\/mydir\u201d to your desired working directory. Step 2: Use glob to match the pattern \u2018csv\u2019 Match the pattern (\u2018csv\u2019) and save the list of file names in the \u2018all_filenames\u2019 variable. You can check out&nbsp;this link&nbsp;to learn more about regular expression matching. Step 3: Combine all&hellip;<\/p>\n<p class=\"more-link-p\"><a class=\"more-link\" href=\"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/2019\/09\/13\/how-to-combine-multiple-csv-files-with-8-lines-of-code\/\">Read more &rarr;<\/a><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[1],"tags":[103,104,102,77],"class_list":["post-2217","post","type-post","status-publish","format-standard","hentry","category-bez-kategorii","tag-csv","tag-glob","tag-pandas","tag-python"],"_links":{"self":[{"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/posts\/2217","targetHints":{"allow":["GET"]}}],"collection":[{"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/comments?post=2217"}],"version-history":[{"count":1,"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/posts\/2217\/revisions"}],"predecessor-version":[{"id":2218,"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/posts\/2217\/revisions\/2218"}],"wp:attachment":[{"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/media?parent=2217"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/categories?post=2217"},{"taxonomy":"post_tag","embeddable":true,"href":"http:\/\/u239160.webh.me\/jakisproblem.pl\/index.php\/wp-json\/wp\/v2\/tags?post=2217"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}