In this blog post, I will explain how to install PySpark on Google Colab

Installing Colab

1. Open drive.google.com in your favorite browser.

2. Click on New on top left → Click on More → Connect more apps

3. Search for Colab and install.

Installing PySpark Libraries on Colab

#install Apache Spark 3.0.1 with Hadoop 2.7 from here.
!wget https://archive.apache.org/dist/spark/spark-3.0.0/spark-3.0.0-bin-hadoop2.7.tgz

# Now, we just need to unzip that folder.
!tar -xvzf spark-3.0.0-bin-hadoop2.7.tgz
!pip install findspark


import os
os.environ["SPARK_HOME"] = "/content/spark-3.0.0-bin-hadoop2.7"
import findspark
findspark.init()

Create Spark Session

from pyspark.sql import SparkSession
spark = SparkSession.builder.master("local").appName("RDD Demo").getOrCreate()
sc = spark.sparkContext

Create RDD

data = list(range(1,101))
rdd = sc.parallelize(data)
rdd.take(5)

---

If you like my work and want to support me…

1. I share tips, tricks and insights on #softwareengineering, #dataengineering #cloud #ml on LinkedIn.

2. Do you want to connect with me, I have started mentoring others for career and interviews at 𝐭𝐨𝐩𝐦𝐚𝐭𝐞.𝐢𝐨/𝐧𝐚𝐯𝐞𝐞𝐧𝐩𝐧