美文网首页
PySpark实战语句

PySpark实战语句

作者: 小甜瓜Melon | 来源:发表于2017-08-09 09:36 被阅读0次
    code1
    feature1 = "id, application_id, user_profile_id, amount"
    sql1 = """SELECT %s FROM tb_source_data.loan_applications LIMIT %d"""%(feature1, 3)
    hiveContext.sql(sql1).show(1000, truncate=False)
    

    等价于

    hiveContext.sql("""SELECT %s FROM tb_source_data.loan_applications LIMIT %d"""%("id, application_id, user_profile_id, amount", 3)).show(1000, truncate=False)
    

    输出结果

    +-----+----------------------------+---------------+--------+
    |id   |application_id              |user_profile_id|amount  |
    +-----+----------------------------+---------------+--------+
    |18132|AAAA17071813423573529711111 |17322          |0.0     |
    |18133|BBBBB17071813472976219211111|17323          |100000.0|
    |18134|CCCC17071813490193476111111 |17324          |0.0     |
    +-----+----------------------------+---------------+--------+
    
    code2

    查看一列的数据

    for i in hiveContext.sql("DESC tb_source_data.loan_applications").collect():
        print i[0]+","
    [output]:
    id,
    application_id,
    user_profile_id,
    amount,
    tenor,
    

    相关文章

      网友评论

          本文标题:PySpark实战语句

          本文链接:https://www.haomeiwen.com/subject/dldfrxtx.html