GitHub Repository: mrdbourke/zero-to-mastery-ml
Path: blob/master/section-2-data-science-and-ml-tools/pandas-exercises-solutions.ipynb
⁸⁷⁴ views

Kernel: Python 3

Pandas Practice (solutions)

This notebook offers a set of solutions to different tasks with pandas.

It should be noted there may be more than one different way to answer a question or complete an exercise.

Exercises are based off (and directly taken from) the quick introduction to pandas notebook.

Different tasks will be detailed by comments or text.

For further reference and resources, it's advised to check out the pandas documnetation.

In [1]:

# Import pandas
import pandas as pd

In [2]:

# Create a series of three different colours
colours = pd.Series(["Blue", "Red", "White"])

In [3]:

# View the series of different colours
colours

Out[3]:

   Blue
    Red
  White
dtype: object

In [4]:

# Create a series of three different car types and view it
cars = pd.Series(["BMW", "Toyota", "Honda"])
cars

Out[4]:

     BMW
  Toyota
   Honda
dtype: object

In [5]:

# Combine the Series of cars and colours into a DataFrame
car_data = pd.DataFrame({"Car make": cars,
                         "Colour": colours})
car_data

Out[5]:

In [6]:

# Import "../data/car-sales.csv" and turn it into a DataFrame
car_sales = pd.read_csv("../data/car-sales.csv")
car_sales

Out[6]:

In [7]:

# Export the DataFrame you created to a .csv file
car_sales.to_csv("../data/exported-car-sales.csv")

In [8]:

# Find the different datatypes of the car data DataFrame
car_sales.dtypes

Out[8]:

Make             object
Colour           object
Odometer (KM)     int64
Doors             int64
Price            object
dtype: object

In [9]:

# Describe your current car sales DataFrame using describe()
car_sales.describe()

Out[9]:

In [10]:

# Get information about your DataFrame using info()
car_sales.info()

Out[10]:

<class 'pandas.core.frame.DataFrame'>
RangeIndex: 10 entries, 0 to 9
Data columns (total 5 columns):
Make             10 non-null object
Colour           10 non-null object
Odometer (KM)    10 non-null int64
Doors            10 non-null int64
Price            10 non-null object
dtypes: int64(2), object(3)
memory usage: 528.0+ bytes

What does it show you?

In [11]:

# Create a Series of different numbers and find the mean of them
series = pd.Series([999, 22203, 43920])
series.mean()

Out[11]:

22374.0

In [12]:

# Create a Series of different numbers and find the sum of them
series = pd.Series([482392, 34994, 22])
series.sum()

Out[12]:

517408

In [13]:

# List out all the column names of the car sales DataFrame
car_sales.columns

Out[13]:

Index(['Make', 'Colour', 'Odometer (KM)', 'Doors', 'Price'], dtype='object')

In [14]:

# Find the length of the car sales DataFrame
len(car_sales)

Out[14]:

10

In [15]:

# Show the first 5 rows of the car sales DataFrame
car_sales.head()

Out[15]:

In [16]:

# Show the first 7 rows of the car sales DataFrame
car_sales.head(7)

Out[16]:

In [17]:

# Show the bottom 5 rows of the car sales DataFrame
car_sales.tail()

Out[17]:

In [18]:

# Use .loc to select the row at index 3 of the car sales DataFrame
car_sales.loc[3]

Out[18]:

Make                    BMW
Colour                Black
Odometer (KM)         11179
Doors                     5
Price            $22,000.00
Name: 3, dtype: object

In [19]:

# Use .iloc to select the row at position 3 of the car sales DataFrame
car_sales.iloc[3]

Out[19]:

Make                    BMW
Colour                Black
Odometer (KM)         11179
Doors                     5
Price            $22,000.00
Name: 3, dtype: object

Notice how they're the same? Why do you think this is?

Check the pandas documentation for .loc and .iloc. Think about a different situation each could be used for and try them out.

In [20]:

# Select the "Odometer (KM)" column from the car sales DataFrame
car_sales["Odometer (KM)"]

Out[20]:

  150043
   87899
   32549
   11179
  213095
   99213
   45698
   54738
   60000
   31600
Name: Odometer (KM), dtype: int64

In [21]:

# Find the mean of the "Odometer (KM)" column in the car sales DataFrame
car_sales["Odometer (KM)"].mean()

Out[21]:

78601.4

In [22]:

# Select the rows with over 100,000 kilometers on the Odometer
car_sales[car_sales["Odometer (KM)"] > 100000]

Out[22]:

In [23]:

# Create a crosstab of the Make and Doors columns
pd.crosstab(car_sales["Make"], car_sales["Doors"])

Out[23]:

In [24]:

# Group columns of the car sales DataFrame by the Make column and find the average
car_sales.groupby(["Make"]).mean(numeric_only=True)

Out[24]:

In [25]:

# Import Matplotlib and create a plot of the Odometer column
# Don't forget to use %matplotlib inline
%matplotlib inline
import matplotlib.pyplot as plt
car_sales["Odometer (KM)"].plot()

Out[25]:

<matplotlib.axes._subplots.AxesSubplot at 0x1235dd278>

In [26]:

# Create a histogram of the Odometer column using hist()
car_sales["Odometer (KM)"].hist()

Out[26]:

<matplotlib.axes._subplots.AxesSubplot at 0x123962240>

In [27]:

# Try to plot the Price column using plot()
car_sales["Price"].plot()

Out[27]:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-27-0c83f8ec73bb> in <module>
      1 # Try to plot the Price column using plot()
----> 2 car_sales["Price"].plot()

~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.6/site-packages/pandas/plotting/_core.py in __call__(self, *args, **kwargs)
    792                     data.columns = label_name
    793 
--> 794         return plot_backend.plot(data, kind=kind, **kwargs)
    795 
    796     def line(self, x=None, y=None, **kwargs):
~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.6/site-packages/pandas/plotting/_matplotlib/__init__.py in plot(data, kind, **kwargs)
     60             kwargs["ax"] = getattr(ax, "left_ax", ax)
     61     plot_obj = PLOT_CLASSES[kind](data, **kwargs)
---> 62     plot_obj.generate()
     63     plot_obj.draw()
     64     return plot_obj.result
~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.6/site-packages/pandas/plotting/_matplotlib/core.py in generate(self)
    277     def generate(self):
    278         self._args_adjust()
--> 279         self._compute_plot_data()
    280         self._setup_subplots()
    281         self._make_plot()
~/Desktop/ml-course/zero-to-mastery-ml/env/lib/python3.6/site-packages/pandas/plotting/_matplotlib/core.py in _compute_plot_data(self)
    412         # no non-numeric frames or series allowed
    413         if is_empty:
--> 414             raise TypeError("no numeric data to plot")
    415 
    416         # GH25587: cast ExtensionArray of pandas (IntegerArray, etc.) to
TypeError: no numeric data to plot

Why didn't it work? Can you think of a solution?

You might want to search for "how to convert a pandas string column to numbers".

And if you're still stuck, check out this Stack Overflow question and answer on turning a price column into integers.

See how you can provide the example code there to the problem here.

In [28]:

# Remove the punctuation from price column
car_sales["Price"] = car_sales["Price"].str.replace(r"[\$\,\.]", "", regex=True)

In [29]:

# Check the changes to the price column
car_sales["Price"]

Out[29]:

   400000
   500000
   700000
  2200000
   350000
   450000
   750000
   700000
   625000
   970000
Name: Price, dtype: object

In [30]:

# Remove the two extra zeros at the end of the price column
car_sales["Price"] = car_sales["Price"].str[:-2]

In [31]:

# Check the changes to the Price column
car_sales["Price"]

Out[31]:

   4000
   5000
   7000
  22000
   3500
   4500
   7500
   7000
   6250
   9700
Name: Price, dtype: object

In [32]:

# Change the datatype of the Price column to integers
car_sales["Price"] = car_sales["Price"].astype(int)

In [33]:

# Lower the strings of the Make column
car_sales["Make"].str.lower()

Out[33]:

  toyota
   honda
  toyota
     bmw
  nissan
  toyota
   honda
   honda
  toyota
  nissan
Name: Make, dtype: object

If you check the car sales DataFrame, you'll notice the Make column hasn't been lowered.

How could you make these changes permanent?

Try it out.

In [34]:

# Make lowering the case of the Make column permanent
car_sales["Make"] = car_sales["Make"].str.lower()

In [35]:

# Check the car sales DataFrame
car_sales

Out[35]:

Notice how the Make column stays lowered after reassigning.

Now let's deal with missing data.

In [36]:

# Import the car sales DataFrame with missing data ("../data/car-sales-missing-data.csv")
car_sales_missing = pd.read_csv("../data/car-sales-missing-data.csv")

# Check out the new DataFrame
car_sales_missing

Out[36]:

Notice the missing values are represented as NaN in pandas DataFrames.

Let's try fill them.

In [37]:

# Fill the Odometer column missing values with the mean of the column inplace
car_sales_missing["Odometer"].fillna(car_sales_missing["Odometer"].mean(),
                                     inplace=True)

In [38]:

# View the car sales missing DataFrame and verify the changes
car_sales_missing

Out[38]:

In [39]:

# Remove the rest of the missing data inplace
car_sales_missing.dropna(inplace=True)

In [40]:

# Verify the missing values are removed by viewing the DataFrame
car_sales_missing

Out[40]:

We'll now start to add columns to our DataFrame.

In [41]:

# Create a "Seats" column where every row has a value of 5
car_sales["Seats"] = 5
car_sales

Out[41]:

In [42]:

# Create a column called "Engine Size" with random values between 1.3 and 4.5
# Remember: If you're doing it from a Python list, the list has to be the same length
# as the DataFrame
engine_sizes = [1.3, 4.3, 2.3, 3.3, 3.0, 2.3, 1.4, 1.7, 2.5, 3.1]
car_sales["Engine Size"] = engine_sizes
car_sales

Out[42]:

In [43]:

# Create a column which represents the price of a car per kilometer
# Then view the DataFrame
car_sales["Price per KM"] = car_sales["Price"] / car_sales["Odometer (KM)"]
car_sales

Out[43]:

In [44]:

# Remove the last column you added using .drop()
car_sales = car_sales.drop("Price per KM", axis=1)
car_sales

Out[44]:

In [45]:

# Shuffle the DataFrame using sample() with the frac parameter set to 1
# Save the the shuffled DataFrame to a new variable
car_sales_sampled = car_sales_sampled = car_sales.sample(frac=1)
car_sales_sampled

Out[45]:

Notice how the index numbers get moved around. The sample() function is a great way to get random samples from your DataFrame. It's also another great way to shuffle the rows by setting frac=1.

In [46]:

# Reset the indexes of the shuffled DataFrame
car_sales_sampled.reset_index()

Out[46]:

Notice the index numbers have been changed to have order (start from 0).

In [47]:

# Change the Odometer values from kilometers to miles using a Lambda function
# Then view the DataFrame
car_sales["Odometer (KM)"] = car_sales["Odometer (KM)"].apply(lambda x: x/1.6)
car_sales

Out[47]:

In [48]:

# Change the title of the Odometer (KM) to represent miles instead of kilometers
car_sales = car_sales.rename(columns={"Odometer (KM)": "Odometer (Miles)"})
car_sales

Out[48]:

Extensions

For more exercises, check out the pandas documentation, particularly the 10-minutes to pandas section.

One great exercise would be to retype out the entire section into a Jupyter Notebook of your own.

Get hands-on with the code and see what it does.

The next place you should check out are the top questions and answers on Stack Overflow for pandas. Often, these contain some of the most useful and common pandas functions. Be sure to play around with the different filters!

Finally, always remember, the best way to learn something new to is try it. Make mistakes. Ask questions, get things wrong, take note of the things you do most often. And don't worry if you keep making the same mistake, pandas has many ways to do the same thing and is a big library. So it'll likely take a while before you get the hang of it.

Pandas Practice (solutions)

Extensions

Product

Resources

Company