Search for a command to run...
Articles tagged with #spark
What is Medallion Architecture Medallion Architecture, proposed by Databricks, enhances data management within a data lakehouse framework. It aligns with the principles of Data as a Product (DaaP) and multi-layered data processing, creating a single ...
This article is for those who want to create scala projects with Apache Maven. Step 01 : Launch IntelliJ IDEA's New Project --> Select Maven Archetype Step 02: Click Add button Step 03 Enter the details as shown and click Add Step 04 Enter the Nam...
SparkContext and SparkSession are two important components in Apache Spark, but they serve different purposes. SparkContext SparkContext (sc) is the entry point for interacting with Spark and represents the connection to a Spark cluster. It was the...
ConceptSQLSpark / PySpark SELECTSELECT column(s) FROM table; SELECT * FROM table; | df.select("column(s)") df.select("*") | | DISTINCT | SELECT DISTINCT column(s) FROM table; | df.select("column(s)").distinct() | | WHERE | SELECT column(s) F...
