美文网首页flink
Flink 入门(二):Mac 上搭建 Flink & Flin

Flink 入门(二):Mac 上搭建 Flink & Flin

作者: alexlee666 | 来源:发表于2019-12-02 09:37 被阅读0次

    Flink V1.9 官方中文文档地址:https://ci.apache.org/projects/flink/flink-docs-release-1.9/zh/

    一、 Mac 上安装 Flink

    1.1 确保安装了 JDK 1.8+ 版本

    Flink 的核心模块使用 Java 开发,因此其运行环境依赖 JDK,且 1.8+ 版本。

    查看 java 版本命令:java -version

    $ java -version
    java version "1.8.0_171"
    Java(TM) SE Runtime Environment (build 1.8.0_171-b11)
    Java HotSpot(TM) 64-Bit Server VM (build 25.171-b11, mixed mode)
    
    

    1.2 安装方法

    直接使用 Mac 自带的 Homebrew 工具安装 Flink 即可:
    命令:brew install apache-flink

    # 默认安装最新版的 flink,安装路径默认/usr/local/Cellar/apache-flink/版本号
    brew install apache-flink
    
    ......
    
    ######################################################################    98.5%
    ######################################################################## 100.0%
    🍺  /usr/local/Cellar/apache-flink/1.9.1: 166 files, 277MB, built in 176 minutes 47 seconds
    ==> `brew cleanup` has not been run in 30 days, running now...
    Removing: /Users/ycaha/Library/Caches/Homebrew/telnet--60.high_sierra.bottle.tar.gz... (52.7KB)
    Removing: /Users/ycaha/Library/Caches/Homebrew/Cask/jd-gui--1.6.0.tar... (1.4MB)
    
    

    1.3 测试安装是否成功 ?

    然后测试是否安装成功:
    命令:flink --version 或者 brew info apache-flink

    $ flink --version
    Version: 1.9.1, Commit ID: 4d56de8
    
    

    或者

    $ brew info apache-flink
    apache-flink: stable 1.9.1, HEAD
    Scalable batch and stream data processing
    https://flink.apache.org/
    /usr/local/Cellar/apache-flink/1.9.1 (167 files, 277MB) *
      Built from source on 2019-11-26 at 14:16:21
    From: https://github.com/Homebrew/homebrew-core/blob/master/Formula/apache-flink.rb
    ==> Requirements
    Required: java = 1.8 ✔
    ==> Options
    --HEAD
        Install HEAD version
    ==> Analytics
    install: 1,585 (30 days), 4,592 (90 days), 18,565 (365 days)
    install_on_request: 1,574 (30 days), 4,572 (90 days), 18,411 (365 days)
    build_error: 0 (30 days)
    
    

    二、启动 Flink

    使用 Mac 的 Homebrew 工具安装的 flink 默认在路径/usr/local/Cellar/apache-flink/1.9.1,其中1.9.1 是版本号。通过脚本来启动 flink,在/usr/local/Cellar/apache-flink/1.9.1/libexec/bin/目录下存在很多脚本文件:

    ll
    total 176
    drwxr-xr-x  24 ycaha  admin  768 Nov 26 14:16 ./
    drwxr-xr-x  11 ycaha  admin  352 Nov 26 14:16 ../
    -rwxr-xr-x   1 ycaha  admin  139 Nov 26 14:16 config.sh*
    -rwxr-xr-x   1 ycaha  admin  148 Nov 26 14:16 find-flink-home.sh*
    -rwxr-xr-x   1 ycaha  admin  135 Nov 26 14:16 flink*
    -rwxr-xr-x   1 ycaha  admin  146 Nov 26 14:16 flink-console.sh*
    -rwxr-xr-x   1 ycaha  admin  145 Nov 26 14:16 flink-daemon.sh*
    -rwxr-xr-x   1 ycaha  admin  146 Nov 26 14:16 historyserver.sh*
    -rwxr-xr-x   1 ycaha  admin  143 Nov 26 14:16 jobmanager.sh*
    -rwxr-xr-x   1 ycaha  admin  152 Nov 26 14:16 mesos-appmaster-job.sh*
    -rwxr-xr-x   1 ycaha  admin  148 Nov 26 14:16 mesos-appmaster.sh*
    -rwxr-xr-x   1 ycaha  admin  150 Nov 26 14:16 mesos-taskmanager.sh*
    -rwxr-xr-x   1 ycaha  admin  155 Nov 26 14:16 pyflink-gateway-server.sh*
    -rwxr-xr-x   1 ycaha  admin  146 Nov 26 14:16 pyflink-shell.sh*
    -rwxr-xr-x   1 ycaha  admin  143 Nov 26 14:16 sql-client.sh*
    -rwxr-xr-x   1 ycaha  admin  147 Nov 26 14:16 standalone-job.sh*
    -rwxr-xr-x   1 ycaha  admin  146 Nov 26 14:16 start-cluster.sh*
    -rwxr-xr-x   1 ycaha  admin  150 Nov 26 14:16 start-scala-shell.sh*
    -rwxr-xr-x   1 ycaha  admin  155 Nov 26 14:16 start-zookeeper-quorum.sh*
    -rwxr-xr-x   1 ycaha  admin  145 Nov 26 14:16 stop-cluster.sh*
    -rwxr-xr-x   1 ycaha  admin  154 Nov 26 14:16 stop-zookeeper-quorum.sh*
    -rwxr-xr-x   1 ycaha  admin  144 Nov 26 14:16 taskmanager.sh*
    -rwxr-xr-x   1 ycaha  admin  145 Nov 26 14:16 yarn-session.sh*
    -rwxr-xr-x   1 ycaha  admin  142 Nov 26 14:16 zookeeper.sh*
    
    

    可以通过运行脚本文件 start-cluster.sh 来启动 flink 集群:

    $ sh start-cluster.sh 
    Starting cluster.
    Starting standalonesession daemon on host LMA23004071M.local.
    Starting taskexecutor daemon on host LMA23004071M.local.
    
    

    通过 ps -ef | grep flink命令可以查到运行的 flink 进程。
    启动 Flink 服务后,client 还可以通过 REST接口连接到 flink server,直接在浏览器中输入:http://localhost:8081/#/overview 即可,其中localhost 是 rest.address,8081 是rest.port

    Flink web界面

    rest.addressrest.port的默认值分别为localhost 和 8081。这两个参数可以在配置文件${flink_home}/libexec/conf/flink-conf.yaml 中修改,然后重启 flink server(通过先后执行脚本 start-cluster.shstop-cluster.sh)使得配置生效。

    #==============================================================================
    # Rest & web frontend
    #==============================================================================
    
    # The port to which the REST client connects to. If rest.bind-port has
    # not been specified, then the server will bind to this port as well.
    #
    # rest.port: 8081
    
    # The address to which the REST client will connect to
    #
    #rest.address: 0.0.0.0
    
    

    相关文章

      网友评论

        本文标题:Flink 入门(二):Mac 上搭建 Flink & Flin

        本文链接:https://www.haomeiwen.com/subject/grejwctx.html