https://w.ublo.ro/ 2026-04-07T11:36:00+00:00 https://w.ublo.ro/ https://w.ublo.ro/_media/wiki/dokuwiki.svg text/html 2023-04-21T12:39:28+00:00 Anonymous (anonymous@undisclosed.example.com) https://w.ublo.ro/cheat-sheets/pyspark?rev=1682080768&do=diff PySpark CheatSheet * Initialize Spark session: from pyspark.sql import SparkSession spark = SparkSession.builder.appName(__file__).getOrCreate() * Create a DataFrame: from pyspark.sql import Row from pyspark.sql.types import StructType, StructField. StringType, IntegerType columns = [ "programming_language", "users_count" ] data = [("Python", 100000), ("Java", 70000), ("C++", 50000)] rdd = spark.sparkContext.parallelize(data) # Resilient Distributed Dataset df_v1 = rdd.toDF() # Dat…