美文网首页
1.1 Eclipse大数据开发过程

1.1 Eclipse大数据开发过程

作者: 旺达哒哒哒 | 来源:发表于2018-11-08 10:30 被阅读0次

Eclipse项目开发过程

1.打开eclipse,新建其他项目:

image.png

2.展开 “Maven”并选择“Maven Project”, 然后点击 “Next”:

image.png

3.选择 “Create a simple project” 再点击“Next”:

image.png

4.确认 GroupId, ArtifactId, Verison, 和 Name, 然后点击“Finish”:

image.png

5.打开 pom.xml file 并点击 “pom.xml” 页面。 将以下Maven依赖粘贴进pom中:

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
  <modelVersion>4.0.0</modelVersion>
    
  <groupId>hdp6java.0811</groupId>
  <artifactId>hdp6</artifactId>
  <version>0.0.1-SNAPSHOT</version>
  <name>test0811</name>
  <packaging>jar</packaging>
  
  <properties>
        <project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
        <project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
        <java.version>1.8</java.version>
        <PermGen>512m</PermGen>
        <MaxPermGen>1024m</MaxPermGen>

        <scala.version>2.10.4</scala.version>
        <scala.binary.version>2.10</scala.binary.version>
        <spark.version>1.6.3</spark.version>
        
        <!--TODO Modify Version After public-cupid RELEASED-->
        <cupid.project.version>3.3.2-public</cupid.project.version>

    </properties>    
    <dependencies>
        <!--spark relative deps-->
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-mllib_${scala.binary.version}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-sql_${scala.binary.version}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_${scala.binary.version}</artifactId>
            <version>${spark.version}</version>
            <scope>provided</scope>
        </dependency>
        
        <dependency>
            <groupId>com.aliyun.odps</groupId>
            <artifactId>cupid-sdk</artifactId>
            <version>${cupid.project.version}</version>
            <scope>provided</scope>
        </dependency>
        
        <dependency>
            <groupId>com.aliyun.odps</groupId>
            <artifactId>odps-spark-datasource_${scala.binary.version}</artifactId>
            <version>${cupid.project.version}</version>
        </dependency>
        
        <dependency>
            <groupId>com.aliyun.odps</groupId>
            <artifactId>hadoop-fs-oss</artifactId>
            <version>${cupid.project.version}</version>
        </dependency>
        <dependency>
                <groupId>log4j</groupId>
                <artifactId>log4j</artifactId>
                <version>1.2.17</version>
        </dependency>
                
    </dependencies>

    <build>
        <plugins>                       
            <plugin>
                <artifactId>maven-assembly-plugin</artifactId>
                <!--<version>3.5.4</version>-->
                <configuration>
                <encoding>UTF-8</encoding>
                    <!--这部分可有可无,加上的话则直接生成可运行jar包-->
                    <!--<archive>-->
                        <!--<manifest>-->
                            <!--<mainClass>${exec.mainClass}</mainClass>-->
                        <!--</manifest>-->
                    <!--</archive>-->
                    <descriptorRefs>
                        <descriptorRef>
                            <!-- 必须是这样写 -->
                            jar-with-dependencies
                        </descriptorRef>
                    </descriptorRefs>
                </configuration>
            </plugin>
        </plugins>
    </build>
</project>

6. 保存pom.xml,右击项目,再Mavne选项中,选择updata project。随后,系统会自动链接maven依赖库,下载阿里云的依赖包,下载完成后,确保pom.xml文件中没有错误,即可开始程序的编写。

image.png

相关文章

网友评论

      本文标题:1.1 Eclipse大数据开发过程

      本文链接:https://www.haomeiwen.com/subject/ajbixqtx.html