IntelliJ IDEA 多个Scala项目如何实现相互调用-Spark-About云-梭伦科技

fc013 发表于 2017-2-11 20:18:14

IntelliJ IDEA 多个Scala项目如何实现相互调用

问题导读：

1.怎样用IntelliJ IDEA开发多个项目?
2.开发多个项目过程中有哪些注意事项？3.多个项目之间的依赖关系是怎样的？

http://www.aboutyun.com/static/image/hrline/4.gif

目录1、需求2、注意事项（6点非常重要）3、建立Project和建立module4、项目之前的依赖关系5、代码展示6、项目中POM依赖展示7、运行结果展示
1、需求1、IntelliJ IDEA打开多个项目
2、每个同学开发一个项目，相互之前独立不影响
3、通过一个入口可以调用所有项目类、方法、属性，达到同时开发且检测代码
4、dependency只需要写一份，其余项目不用写，便可全部依赖
2、注意事项（非常重要）6个坑：
1、<groupId>com.yh.bi</groupId>
项目中所有的groupId要一样
2、避免循环依赖，导致程序报错
3、<scope>provided</scope>
打包的服务器运行时候需要provided，本机调试的时候，需要注释
在一个maven项目中，如果存在编译需要而发布不需要的jar包，可以用scope标签，值设为provided

4、项目、module建好之后需要添加Scala的框架支持

5、在yhProject中，可以统一对所有的module进行清理、编译、打包

6、要运行依赖中的module，则必须要将module中的Jar包，打到maven中，需要使用install
下面，是我将所有module中的Jar打到Maven中的路径：

3、建立Project和建立module1、只需要建立一个项目，其他项目由module建立，所有module且放在项目中。
2、本文项目为yhproject，其余都为module，分别是：mainEntrance、yhutils、yhapp、yhweb、yhgame项目建立步鄹：

Module建立步鄹：

项目、所有module、部分module代码展示：

4、项目之前的依赖关系

5、代码展示mainEntrance
package com.yh.bi.dag
package com.yh.bi.dag

/**
* Created by yuhui on 2017/2/10.
*/
import java.time.{Duration, LocalDate}
import com.yh.bi._
import com.yh.bi.{UserAPP, UserGame, UserWEB}
import org.slf4j.LoggerFactory

import scala.collection.immutable.{ListMap, ListSet}

/**
* Created by yuhui on 2016/8/25.
* task --> Node --> DAG --> DAGExecutor
*/

case class Node(task: T, parent: T*) {
override def toString: String = {
s"$task(${parent.mkString(",")})"
}
}

case class DAG(nodes: Node*)

case class DAGExecutor(dag: DAG) {
private val LOG = LoggerFactory.getLogger(this.getClass)
private val _nodes: Map] = dag.nodes.map(node => (node.task, node.parent.filter(_ != null))).toMap
private var _pending: Set = ListSet()
private var _fails = ListMap()
private var _success = Seq()

//判断Node的task节点的父节点运行状态（flase ，true）
private def getPending: Option = {
_pending.find { name =>
   val parents = _nodes(name)
   !parents.exists(name => !_success.contains(name))
}
}

private def fail(name: T, message: String): Unit = {
_pending -= name
_fails += name -> message
for (child <- _pending.filter(child => _nodes(child).contains(name))) {
   fail(child, s"依赖的任务无法执行: $name")
}
}

private def success(name: T): Unit = {
_pending -= name
_success = _success :+ name
}

def execute(func: T => Unit): Unit = {
_pending = _nodes.keySet
_fails = ListMap()
_success = Seq()
var running = true

while (running) {
   val taskOpt = getPending
   if (taskOpt.nonEmpty) {
   val task = taskOpt.get
   val startMills = System.currentTimeMillis()
   LOG.info("start task {}", task)
   try {
      println("=============")
      func(task)//执行executor方法
      println("+++++++++++++")
      val time = Duration.ofMillis(System.currentTimeMillis() - startMills)
      LOG.info(s"end task $task time=$time")
      success(task)
   } catch {
      case e: Throwable => fail(task, e.getMessage)
         LOG.error(e.getMessage, e)
         LOG.info(s"fail task $task")
   }
   } else {
   running = false
   }
}

for (name <- _success) {
   LOG.info(s"success task: $name")
}
for (name <- _fails) {
   LOG.info(s"fail task: ${name._1} - ${name._2}")
}
}
}

object DAG {
val allSDKDAG = new DAG(
Node(UserAPP),
Node(UserGame),
Node(UserWEB)
)

def main(args: Array): Unit = {
DAGExecutor(allSDKDAG).execute { task =>task.executor("appkey": String, LocalDate.now(), LocalDate.now())}
}
}
yhutils
com.yh.bi
package com.yh.bi

/**
* Created by yuhui on 2017/2/10.
*/
import java.time.LocalDate
import org.apache.spark.sql.SQLContext
import org.slf4j.LoggerFactory

abstract class Executor extends Task with SQLContextAware {

override def run(appkey: String, startDay: LocalDate, endDay: LocalDate)={}

}

trait SQLContextAware {
implicit var ctx: SQLContext = _
}

abstract class Task {

protected val LOG = LoggerFactory.getLogger(this.getClass)

def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit

def run(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = {
executor(appkey, startDay, endDay)
}

}
yhapp
com.yh.bi
package com.yh.bi

/**
* Created by yuhui on 2017/2/10.
*/
import java.time.LocalDate

object UserAPP extends Executor{

override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = {

println("++++我的UserAPP的执行过程++++")

}

}
yhweb
com.yh.bi
package com.yh.bi

import java.time.LocalDate

object UserWEB extends Executor{

override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = {

println("++++我的UserWEB的执行过程++++")

}

}
yhgame
com.yh.bi
package com.yh.bi

/**
* Created by yuhui on 2017/2/10.
*/
import java.time.LocalDate

object UserGame extends Executor{

override def executor(appkey: String, startDay: LocalDate, endDay: LocalDate): Unit = {

println("++++我的UserGame的执行过程++++")

}

}
6、项目中POM依赖展示yhproject中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>
<groupId>com.yh.bi</groupId>
<artifactId>yhproject</artifactId>
<packaging>pom</packaging>
<version>1.0</version>

<modules>
   <module>mainEntrance</module>
   <module>yhapp</module>
   <module>yhweb</module>
   <module>yhgame</module>
   <module>yhutils</module>
</modules>
</project>
mainEntrance中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<modelVersion>4.0.0</modelVersion>
<artifactId>mainEntrance</artifactId>
<groupId>com.yh.bi</groupId>
<version>1.0</version>

<dependencies>
   <dependency>
         <artifactId>yhutils</artifactId>
         <groupId>com.yh.bi</groupId>
         <version>1.0</version>
         
   </dependency>

   <dependency>
         <artifactId>yhapp</artifactId>
         <groupId>com.yh.bi</groupId>
         <version>1.0</version>
         
   </dependency>

   <dependency>
         <artifactId>yhgame</artifactId>
         <groupId>com.yh.bi</groupId>
         <version>1.0</version>
         
   </dependency>

   <dependency>
         <artifactId>yhweb</artifactId>
         <groupId>com.yh.bi</groupId>
         <version>1.0</version>
         
   </dependency>
</dependencies>

</project>
yhutils中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
<parent>
   <artifactId>yhproject</artifactId>
   <groupId>com.yh.bi</groupId>
   <version>1.0</version>
</parent>

<modelVersion>4.0.0</modelVersion>
<artifactId>yhutils</artifactId>

<dependencies>
   <dependency>
         <groupId>org.apache.spark</groupId>
         <artifactId>spark-hive_2.11</artifactId>
         <version>1.6.1</version>
   </dependency>
</dependencies>
</project>
yhapp中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>
<groupId>com.yh.bi</groupId>
<artifactId>yhapp</artifactId>
<version>1.0</version>

<dependencies>
   <dependency>
         <artifactId>yhutils</artifactId>
         <groupId>com.yh.bi</groupId>
         <version>1.0</version>
         <scope>provided</scope>
   </dependency>
</dependencies>

</project>
yhweb中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>
<groupId>com.yh.bi</groupId>
<artifactId>yhweb</artifactId>
<version>1.0</version>

<dependencies>
   <dependency>
         <artifactId>yhutils</artifactId>
         <groupId>com.yh.bi</groupId>
         <version>1.0</version>
         <scope>provided</scope>
   </dependency>
</dependencies>

</project>
yhgame中POM文件展示：
<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">

<modelVersion>4.0.0</modelVersion>
<groupId>com.yh.bi</groupId>
<artifactId>yhgame</artifactId>
<version>1.0</version>

<dependencies>
   <dependency>
         <artifactId>yhutils</artifactId>
         <groupId>com.yh.bi</groupId>
         <version>1.0</version>
         <scope>provided</scope>
   </dependency>
</dependencies>
</project>
7、运行结果展示注意：我在mainEntrance执行DAG中的main文件，可以调用且执行了yhutils、yhapp、yhweb、yhgame中的代码

来源：csdn作者：silentwolfyh

页: [1]

About云-梭伦科技's Archiver

IntelliJ IDEA 多个Scala项目如何实现相互调用