pyspark

作者: 叫兽吃橙子 | 来源:发表于2019-04-03 11:03 被阅读0次

mysql = '''
select * from app.dj_live_entire_stat_di where ds='2019-02-01' and ptype='m' and stat_type='all'
'''
df = spark.sql(mysql)
df.show()

df_m =  spark.read.parquet('/user/hive/warehouse/app.db/dj_live_entire_stat_di/stat_type=all/ptype=m/ds=2019-02-01/000000_0')
df_m.dtypes

Spark Python API Docs(part one)
pyspark package subpackages pyspark.sql module pyspark.st...
pyspark整理
pyspark入门资料公众号回复：pyspark （会有pyspark资料大礼包：Learning PySpar...
PySpark初见
PySpark PySpark 是 Spark 为 Python 开发者提供的 API。子模块pyspark.s...
Jupyter配置教程
将jupyter notebook作为pyspark的默认编辑器安装pyspark通过拷贝pyspark包安装源...
PySpark Recipes A Problem-Soluti
PySpark Recipes A Problem-Solution Approach with PySpark2...
Spark-pyspark
pyspark介绍 pyspark是Spark官方提供的API接口，同时pyspark也是Spark中的一个程序。...
Spark Python API Docs(part three
pyspark.streaming module Module contents class pyspark.st...
from pyspark.mllib.recommendatio
进入pyspark输入from pyspark.mllib.recommendation import Ratin...
pyspark读写hbase
安装pyspark pip install pyspark==2..3.2 --default-timeout=1...
PySpark(用户定义函数)
来源：https://sparkbyexamples.com/pyspark/pyspark-udf-user-d...