美文网首页
1.1 Eclipse大数据开发过程

1.1 Eclipse大数据开发过程

作者: 旺达哒哒哒 | 来源:发表于2018-11-08 10:30 被阅读0次

    Eclipse项目开发过程

    1.打开eclipse,新建其他项目:

    image.png

    2.展开 “Maven”并选择“Maven Project”, 然后点击 “Next”:

    image.png

    3.选择 “Create a simple project” 再点击“Next”:

    image.png

    4.确认 GroupId, ArtifactId, Verison, 和 Name, 然后点击“Finish”:

    image.png

    5.打开 pom.xml file 并点击 “pom.xml” 页面。 将以下Maven依赖粘贴进pom中:

    <?xml version="1.0" encoding="UTF-8"?>
    <project xmlns="http://maven.apache.org/POM/4.0.0"
             xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
             xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
      <modelVersion>4.0.0</modelVersion>
        
      <groupId>hdp6java.0811</groupId>
      <artifactId>hdp6</artifactId>
      <version>0.0.1-SNAPSHOT</version>
      <name>test0811</name>
      <packaging>jar</packaging>
      
      <properties>
            <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
            <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
            <java.version>1.8</java.version>
            <PermGen>512m</PermGen>
            <MaxPermGen>1024m</MaxPermGen>
    
            <scala.version>2.10.4</scala.version>
            <scala.binary.version>2.10</scala.binary.version>
            <spark.version>1.6.3</spark.version>
            
            <!--TODO Modify Version After public-cupid RELEASED-->
            <cupid.project.version>3.3.2-public</cupid.project.version>
    
        </properties>    
        <dependencies>
            <!--spark relative deps-->
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-mllib_${scala.binary.version}</artifactId>
                <version>${spark.version}</version>
                <scope>provided</scope>
            </dependency>
            
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-sql_${scala.binary.version}</artifactId>
                <version>${spark.version}</version>
                <scope>provided</scope>
            </dependency>
            
            <dependency>
                <groupId>org.apache.spark</groupId>
                <artifactId>spark-core_${scala.binary.version}</artifactId>
                <version>${spark.version}</version>
                <scope>provided</scope>
            </dependency>
            
            <dependency>
                <groupId>com.aliyun.odps</groupId>
                <artifactId>cupid-sdk</artifactId>
                <version>${cupid.project.version}</version>
                <scope>provided</scope>
            </dependency>
            
            <dependency>
                <groupId>com.aliyun.odps</groupId>
                <artifactId>odps-spark-datasource_${scala.binary.version}</artifactId>
                <version>${cupid.project.version}</version>
            </dependency>
            
            <dependency>
                <groupId>com.aliyun.odps</groupId>
                <artifactId>hadoop-fs-oss</artifactId>
                <version>${cupid.project.version}</version>
            </dependency>
            <dependency>
                    <groupId>log4j</groupId>
                    <artifactId>log4j</artifactId>
                    <version>1.2.17</version>
            </dependency>
                    
        </dependencies>
    
        <build>
            <plugins>                       
                <plugin>
                    <artifactId>maven-assembly-plugin</artifactId>
                    <!--<version>3.5.4</version>-->
                    <configuration>
                    <encoding>UTF-8</encoding>
                        <!--这部分可有可无,加上的话则直接生成可运行jar包-->
                        <!--<archive>-->
                            <!--<manifest>-->
                                <!--<mainClass>${exec.mainClass}</mainClass>-->
                            <!--</manifest>-->
                        <!--</archive>-->
                        <descriptorRefs>
                            <descriptorRef>
                                <!-- 必须是这样写 -->
                                jar-with-dependencies
                            </descriptorRef>
                        </descriptorRefs>
                    </configuration>
                </plugin>
            </plugins>
        </build>
    </project>
    

    6. 保存pom.xml,右击项目,再Mavne选项中,选择updata project。随后,系统会自动链接maven依赖库,下载阿里云的依赖包,下载完成后,确保pom.xml文件中没有错误,即可开始程序的编写。

    image.png

    相关文章

      网友评论

          本文标题:1.1 Eclipse大数据开发过程

          本文链接:https://www.haomeiwen.com/subject/ajbixqtx.html