{ "componentChunkName": "component---src-templates-blog-post-js", "path": "/combine-multiple-dataframes-using-the-concat-method/", "result": {"data":{"markdownRemark":{"html":"

Let’s read 3 Excel files into 3 DataFrames:

\n
import pandas as pd\n\nsalesJan = pd.read_excel("salesJan.xlsx")\n\nsalesFeb = pd.read_excel("salesFeb.xlsx")\n\nsalesMarch = pd.read_excel("salesMarch.xlsx")
\n

Let’s preview these DataFrames:

\n
salesJan
\n

Output:

\n

\n \n \n \n \n

\n
salesMarch
\n

Output:

\n

\n \n \n \n \n

\n

Notice that all three DataFrames have the same three columns (Store, City, Sales), and the columns are in the same order.

\n

We can combine the 3 DataFrames using the concat() method by passing them inside a Python List.

\n
pd.concat([salesJan, salesFeb, salesMarch])
\n

Output:

\n

\n \n \n \n \n

\n

As highlighted above, we can see that we have combined the indexes as well. We can ignore these indexes and instead create a brand new one by passing ignore_index = True to the concat() method.

\n
salesQ4 = pd.concat([salesJan, salesFeb, salesMarch], ignore_index = True)\n\nsalesQ4
\n

Output:

\n

\n \n \n \n \n

\n

We have successfully combined the DataFrames; however, we have got a problem.

\n

The problem is: in the combined salesQ4 DataFrame, we can’t identify which sales figures belong to which DataFrame.

\n

We can solve this problem by passing a List to the keys parameter. The List items will correspond to their respective DataFrames. (We can name the List items anything we want).

\n
salesQ4 = pd.concat([salesJan, salesFeb, salesMarch], keys=["January", "February", "March"])\n\nsalesQ4
\n

Output:

\n

\n \n \n \n \n

\n

The result is a multi-index DataFrame.

\n

Now consider a scenario where the columns in the DataFrames are not in the same order.

\n

I will reorder the columns in the salesJan & salesFeb Excel files. The order in the salesMarch file remains the same.

\n

Let’s read the Excel files into DataFrames:

\n
import pandas as pd\n\nsalesJanMod = pd.read_excel("salesJanMod.xlsx")\n\nsalesFebMod = pd.read_excel("salesFebMod.xlsx")\n\nsalesMarch = pd.read_excel("salesMarch.xlsx")
\n

Let’s preview these DataFrames:

\n
salesJanMod
\n

Output:

\n

\n \n \n \n \n

\n
salesFebMod
\n

Output:

\n

\n \n \n \n \n

\n

Let’s combine these DataFrames:

\n
pd.concat([salesMarch, salesFebMod, salesJanMod,], keys=["Mar", "Feb", "Jan"]).fillna(value="N/A")
\n

Output:

\n

\n \n \n \n \n

\n

Even though the columns were in different order, Pandas did manage to combine the 3 DataFrames without any problem.

\n

We can combine the DataFrames horizontally as well by passing axis = “Columns” to the concat() method.

\n
pd.concat([salesJanMod, salesFebMod, salesMarch], keys=["Jan", "Feb", "March"], axis="columns")
\n

Output:

\n

\n \n .grvsc-code > .grvsc-line::before {\n content: ' ';\n position: absolute;\n width: 100%;\n }\n \n .grvsc-line-diff-add::before {\n background-color: var(--grvsc-line-diff-add-background-color, rgba(0, 255, 60, 0.2));\n }\n \n .grvsc-line-diff-del::before {\n background-color: var(--grvsc-line-diff-del-background-color, rgba(255, 0, 20, 0.2));\n }\n \n .grvsc-line-number {\n padding: 0 2px;\n text-align: right;\n opacity: 0.7;\n }\n \n .synthwave-84 { background-color: #262335; }\n .synthwave-84 .mtki { font-style: italic; }\n .synthwave-84 .mtk10 { color: #FEDE5D; }\n .synthwave-84 .mtk15 { color: #FF7EDBFF; }\n .synthwave-84 .mtk12 { color: #FFFFFFEE; }\n .synthwave-84 .mtk6 { color: #36F9F6; }\n .synthwave-84 .mtk16 { color: #FF8B39; }\n .synthwave-84 .mtk8 { color: #72F1B8; }\n .synthwave-84 .mtk5 { color: #F97E72; }\n .synthwave-84 .grvsc-line-highlighted::before {\n background-color: var(--grvsc-line-highlighted-background-color, rgba(255, 255, 255, 0.1));\n box-shadow: inset var(--grvsc-line-highlighted-border-width, 4px) 0 0 0 var(--grvsc-line-highlighted-border-color, rgba(255, 255, 255, 0.5));\n }\n","frontmatter":{"title":"Combine Multiple DataFrames Using The Concat() Method","date":"2021-08-09"}}},"pageContext":{"slug":"/combine-multiple-dataframes-using-the-concat-method/","prev":{"fields":{"slug":"/using-the-in-keyword-to-check-whether-a-value-exists-in-a-pandas-series/"},"frontmatter":{"modules":null}},"next":{"fields":{"slug":"/creating-spreadsheet-style-pivot-tables-in-pandas/"},"frontmatter":{"modules":null}}}}, "staticQueryHashes": ["3159585216"]}