Getting Started with the most popular Data Science Library: Pandas, Lecture 2

Data structures in pandas

Data Science

The term data science can sound pretty interesting to someone experienced with programming and adept at statistics and econometrics, or it may sound hugely intimidating to someone who is a newbie to software and has no prior quantitative knowledge. But it may be made intelligible and penetrable to anyone and everyone with a passion to learn data science. Broadly speaking, data science constitutes of the following components:

  1. Modelling
  2. Computer software
  3. Statistics
  4. Econometrics
  5. Mathematics
  6. Analytics

At its core, data science involves using automated methods to analyze massive amounts of data and from different fields i.e., mobile sensors, sophisticated instruments, the web and more, is keeping on increasing. Such a future is not far away where we will see data science transforming everything, from healthcare to media. Python is really a great tool, and there are several reasons why Data Scientists like python. Data scientists need to use data visualizations to clearly communicate outputs and predictions to stakeholders at any level of a business. This is the real value a great data scientist can provide – without this, it’s a zero-sum game. 


One of the most frequently used libraries for data science in Python is Pandas, and also one of the most crucial for data science in Python.  Pandas stands for “Python Data Analysis Library. Pandas is an open-source Python Library providing high-performance data manipulation and analysis tool using its powerful data structures. The name Pandas is derived from the word Panel Data – an Econometrics from Multidimensional data.

In 2008, developer Wes McKinney started developing pandas when in need of high performance, flexible tool for analysis of data. The Pandas module is a high performance, highly efficient, and high-level data analysis library. With pandas one can easily load in, and output out in the .xls or .xlsx format quickly, so, even if the client wants to view things the old way, they can. Pandas is also compatible with text files, csv, xlsx, html, and more with its incredibly powerful IO. The highlight of pandas is lightning fast data analysis on spreadsheet-like data, with an extremely robust input/output mechanism for handling multiple data types and even converting to and from data types.

Pandas is quite a game changer when it comes to analyzing data with Python and it is one of the most preferred and widely used tools in data munging/wrangling if not the most used one.

What’s cool about Pandas is that it takes data (like a CSV or TSV file, or a SQL database) and creates a Python object with rows and columns called data frame that looks very similar to table in a statistical software (think Excel or SPSS for example. People who are familiar with R would see similarities to R too). This is so much easier to work with in comparison to working with lists and/or dictionaries through for loops or list comprehension.

Prior to Pandas, Python was majorly used for data munging and preparation. It had very little contribution towards data analysis. Pandas solved this problem. Using Pandas, we can accomplish five typical steps in the processing and analysis of data, regardless of the origin of data — load, prepare, manipulate, model, and analyze. Python with Pandas is used in a wide range of fields including academic and commercial domains including finance, economics, Statistics, analytics, etc.

Key Features of Pandas

  • Fast and efficient DataFrame object with default and customized indexing.
  • Tools for loading data into in-memory data objects from different file formats.
  • Data alignment and integrated handling of missing data.
  • Reshaping and pivoting of date sets.
  • Label-based slicing, indexing and subsetting of large data sets.
  • Columns from a data structure can be deleted or inserted.
  • Group by data for aggregation and transformations.
  • High performance merging and joining of data.
  • Time Series functionality.



WhatsApp chat