[图片上传失败...(image-47bff3-1555426723726)]
[图片上传失败...(image-23a1a9-1555426723726)]
[图片上传失败...(image-a9f209-1555426723726)]
[图片上传失败...(image-16fb99-1555426723726)]

Piggy Metrics

个人财务的简易解决方案

This is a proof-of-concept application, which demonstrates Microservice Architecture Pattern using Spring Boot, Spring Cloud and Docker。
With a pretty neat user interface, by the way。

[图片上传失败...(image-49ef69-1555426723726)]
[图片上传失败...(image-984dbb-1555426723726)]

业务服务

PiggyMetrics 分解为三个核心微服务。 全部都是独立可部署应用, 根据各自的业务域进行编排。

<img width="880" alt="Functional services" src="https://cloud。githubusercontent。com/assets/6069066/13900465/730f2922-ee20-11e5-8df0-e7b51c668847。png">

账户服务

涵盖了通用用户登录逻辑以及验证: 收入/支持 项目, 储蓄以及账户设置。

方法	路径	备注	用户验证	用户界面
GET	/accounts/{account}	获取指定账号信息
GET	/accounts/current	获取当前账户信息	×	×
GET	/accounts/demo	获取演示账户信息 (收入/支出信息, 等)		×
PUT	/accounts/current	保存当前账户信息	×	×
POST	/accounts/	注册新账号		×

统计信息

对每个账号的主要统计数据进行计算，并为捕获时序。( 不知道咋翻译 译者注 )数据点包含值，标准化为基础货币和时间段。此数据用于跟踪帐户生命周期中的现金流动态。

方法	路径	备注	用户验证	用户界面
GET	/statistics/{account}	指定账户的统计信息
GET	/statistics/current	获取当前统计信息	×	×
GET	/statistics/demo	获取演示账户统计信息		×
PUT	/statistics/{account}	创建或更新指定账户的时序数据

通知服务

存储了用户通讯录信息以及通知设置 (譬如提醒和备份频率)。
定时任务从其他服务收集了需要的信息以及发送邮件消息到订阅用户。

方法	路径	备注	用户验证	用户界面
GET	/notifications/settings/current	获取当前用户通知设置	×	×
PUT	/notifications/settings/current	保存当前用户通知设置	×	×

备注

• 每个微服务都有自己的数据库, 导致没有办法绕过 API 以及直接访问持久化数据
• 这个项目, 我使用了MongoDB 作为每个微服务的主要数据库。
  这对于这种多语言 持久化架构是有好处的(译的有点奇怪 译者注) (选择这个类型的数据库是最好的!!)。
• 服务与服务间的通信比较简单: 微服务交互只使用异步Restful API。
  现实系统中的常见做法是使用交互样式的组合。
  例如, 使用异步 GET 请求去检索数据以及异步去通过消息代理进行创建/更新操作，以分离服务和缓冲消息。但是，这将使我们进入结果一致性世界。

基础服务

分布式系统中有许多常见的模式，可以帮助我们使所描述的核心服务工作起来。[Spring Cloud]（http://projects.spring.io/spring-cloud/）提供了增强Spring引导应用程序行为以实现这些模式的强大工具。我将简要介绍它们。
<img width="880" alt="Infrastructure services" src="https://cloud。githubusercontent。com/assets/6069066/13906840/365c0d94-eefa-11e5-90ad-9d74804ca412。png">

配置中心 服务

Spring Cloud Config 是用于分布式系统的水平可扩展的集中配置服务。它使用可插入的存储库层，当前支持本地存储、Git和Subversion。

在这个项目中，我使用 native profile, 它只从本地类路径加载配置文件。 你可以在 Config service resources 看到 shared 目录。
现在，当通知服务请求其配置时，使用shared/notification-service。yml 和 shared/application。yml （在所有客户端应用程序之间共享）。

客户端配置

只需只用 spring-cloud-starter-config 依赖, 自动配置就可以完成剩下的了。

现在，您不需要在应用程序中嵌入任何属性。 只需要提供 bootstrap.yml 应用名和配置中心地址:

spring:
  application:
    name: notification-service
  cloud:
    config:
      uri: http://config:8888
      fail-fast: true

使用 Spring Cloud Config, 可以动态的切换应用的配置。

例如, [EmailService bean](https://github.com/jinweibin/PiggyMetrics/blob/master/notification-service/src/main/java/com
/piggymetrics/notification/service/EmailServiceImpl.java) 使用 @RefreshScope 注解。
这就意味着不需要重启以及重新编译的情况就可以通知应用服务变更电子邮件内容和副标题。

首先将配置中心修改参数,然后,发送刷新请求以通知服务参数更新:
curl -H "Authorization: Bearer #token#" -XPOST http://127.0.0.1:8000/notifications/refresh

另外, 你也可以使用 Git 的 Webhooks webhooks to automate this process

备注

• 动态刷新有一些限制。 @RefreshScope 在有 @Configuration 注解的类不支持还有处理不了有 @Scheduled 注解的方法
• fail-fast 属性表示如果在不能连接配置中心的时候会在启动时马上失败。
• 这里有些重要的笔记 security notes 下

鉴权服务

鉴权任务被分摊到各个微服务上,那些被 OAuth2 tokens 授权的后台服务资源。
Auth Server is used for user authorization as well as for secure machine-to-machine communication inside a perimeter。
鉴权服务器用于用户鉴权,也用于在外围环境中进行安全的机器到机器通信。。

这个项目用户鉴权方式使用的是 Password credentials 授权方式
(因为他只给本地Piggmetrics用户界面使用) ,另外微服务的授权使用 Client Credentials 授权。

Spring Cloud Security 提供了方便的注解以及自动配置使应用能够更加简单的实现服务端以及客户端的鉴权 。
在这里你可以学到更多 文档 也可以在 Auth Server code确认详细配置。

对于客户端, 所有的鉴权工作都和原来基于 session 的鉴权方式一样, 你可以在 request 中获取 Principal 对象, 基于表达式和@PreAuthorize注解去验证用户的角色或者其他内容
每个PiggyMetrics的客户端(账户服务,统计服务,通知服务和浏览器)后端服务都拥有server作用域,浏览器则拥有ui。
所以我们也可以保护控制器不受外部访问的影响, 例如:

@PreAuthorize("#oauth2。hasScope('server')")
@RequestMapping(value = "accounts/{name}", method = RequestMethod.GET)
public List<DataPoint> getStatisticsByAccountName(@PathVariable String name) {
    return statisticsService.findByAccountName(name);
}

API 网关

如你所见, 这边有3个核心服务,他们向其他的客户端暴露外部API接口。
在真实系统中,这个数字会随着系统的复杂性增长得非常之快。
事实上, 为了渲染一个复杂的网页可能要触发上百上千个服务。

理论上, 客户端可以直接发送请求到各个微服务供应商去。
但是很明显的问题是, 这个操作会有很大的挑战以及限制, 像是必须知道所有节点的地址, 分别对每一条信息执行HTTP请求, 然后在一个客户端去合并结果。
另一个问题是后端可能使用的是非Web友好协议。

通常来说, 使用 API 网关可能是一个更好的方法。
It is a single entry point into the system, used to handle requests by routing them to the appropriate backend service or by invoking multiple backend services and aggregating the results。
这样进入系统就只有一个入口, 可以通过将请求路由到适合的后端服务或者多个好多服务aggregating the results。
此外，它还可以用于身份验证、监控、压力和金丝雀测试、服务迁移、静态响应处理、主动流量管理。

Netflix 开源了 这样的边缘服务,
现在我们就可以使用 Spring Cloud 的@EnableZuulProxy 注解去开启它。
In this project, I use Zuul to store static content (ui application) and to route requests to appropriate
这个项目里, 我使用了 Zuul 去存储静态资源内容 ( 用户界面应用 ) 还有去路由请求到合适的微服务去。
Here's a simple prefix-based routing configuration for Notification service:
这里是一个简单的基于前缀的通知服务的路由配置:

zuul:
  routes:
    notification-service:
        path: /notifications/**
        serviceId: notification-service
        stripPrefix: false

以上配置以为着所有以 /notifications 开头的请求都会被路由到通知服务去。
这边没有往常的硬编码的地址。 Zuul 使用了 服务发现
机制去定位通知服务的所有实例然后 [负载均衡](https://github.com/jinweibin/PiggyMetrics/blob/master/README
.md#http-client-load-balancer-and-circuit-breaker)。

服务发现

另一种常见的架构模式是服务发现。
这可以自动检测到服务实例的网络位置,
它可以根据服务的故障,升级或者是自动伸缩来动态的分配地址。

服务发现的关键就是注册中心。
这个项目使用了Netflix Eureka 作为服务的注册中心。
Eureka is a good example of the client-side discovery pattern,
Eureka 是一个很好的客户端发现模式的例子,
when client is responsible for determining locations of available service instances (using Registry server) and load balancing requests across them。

With Spring Boot, you can easily build Eureka Registry with spring-cloud-starter-eureka-server dependency, @EnableEurekaServer annotation and simple configuration properties。

Client support enabled with @EnableDiscoveryClient annotation an bootstrap。yml with application name:

spring:
  application:
    name: notification-service

Now, on application startup, it will register with Eureka Server and provide meta-data, such as host and port, health indicator URL, home page etc。 Eureka receives heartbeat messages from each instance belonging to a service。 If the heartbeat fails over a configurable timetable, the instance will be removed from the registry。

Also, Eureka provides a simple interface, where you can track running services and a number of available instances: http://localhost:8761

Load balancer, Circuit breaker and Http client

Netflix OSS provides another great set of tools。

Ribbon

Ribbon is a client side load balancer which gives you a lot of control over the behaviour of HTTP and TCP clients。 Compared to a traditional load balancer, there is no need in additional hop for every over-the-wire invocation - you can contact desired service directly。

Out of the box, it natively integrates with Spring Cloud and Service Discovery。 Eureka Client provides a dynamic list of available servers so Ribbon could balance between them。

Hystrix

Hystrix is the implementation of Circuit Breaker pattern, which gives a control over latency and failure from dependencies accessed over the network。 The main idea is to stop cascading failures in a distributed environment with a large number of microservices。 That helps to fail fast and recover as soon as possible - important aspects of fault-tolerant systems that self-heal。

Besides circuit breaker control, with Hystrix you can add a fallback method that will be called to obtain a default value in case the main command fails。

Moreover, Hystrix generates metrics on execution outcomes and latency for each command, that we can use to monitor system behavior。

Feign

Feign is a declarative Http client, which seamlessly integrates with Ribbon and Hystrix。 Actually, with one spring-cloud-starter-feign dependency and @EnableFeignClients annotation you have a full set of Load balancer, Circuit breaker and Http client with sensible ready-to-go default configuration。

Here is an example from Account Service:

@FeignClient(name = "statistics-service")
public interface StatisticsServiceClient {

    @RequestMapping(method = RequestMethod。PUT, value = "/statistics/{accountName}", consumes = MediaType。APPLICATION_JSON_UTF8_VALUE)
    void updateStatistics(@PathVariable("accountName") String accountName, Account account);

}

• Everything you need is just an interface
• You can share @RequestMapping part between Spring MVC controller and Feign methods
• Above example specifies just desired service id - statistics-service, thanks to autodiscovery through Eureka (but obviously you can access any resource with a specific url)

Monitor dashboard

In this project configuration, each microservice with Hystrix on board pushes metrics to Turbine via Spring Cloud Bus (with AMQP broker)。 The Monitoring project is just a small Spring boot application with Turbine and Hystrix Dashboard。

See below how to get it up and running。

Let's see our system behavior under load: Account service calls Statistics service and it responses with a vary imitation delay。 Response timeout threshold is set to 1 second。

<img width="880" src="https://cloud。githubusercontent。com/assets/6069066/14194375/d9a2dd80-f7be-11e5-8bcc-9a2fce753cfe。png">

<img width="212" src="https://cloud。githubusercontent。com/assets/6069066/14127349/21e90026-f628-11e5-83f1-60108cb33490。gif">	<img width="212" src="https://cloud。githubusercontent。com/assets/6069066/14127348/21e6ed40-f628-11e5-9fa4-ed527bf35129。gif">	<img width="212" src="https://cloud。githubusercontent。com/assets/6069066/14127346/21b9aaa6-f628-11e5-9bba-aaccab60fd69。gif">	<img width="212" src="https://cloud。githubusercontent。com/assets/6069066/14127350/21eafe1c-f628-11e5-8ccd-a6b6873c046a。gif">
0 ms delay	500 ms delay	800 ms delay	1100 ms delay
Well behaving system。 The throughput is about 22 requests/second。 Small number of active threads in Statistics service。 The median service time is about 50 ms。	The number of active threads is growing。 We can see purple number of thread-pool rejections and therefore about 30-40% of errors, but circuit is still closed。	Half-open state: the ratio of failed commands is more than 50%, the circuit breaker kicks in。 After sleep window amount of time, the next request is let through。	100 percent of the requests fail。 The circuit is now permanently open。 Retry after sleep time won't close circuit again, because the single request is too slow。

Log analysis

Centralized logging can be very useful when attempting to identify problems in a distributed environment。 Elasticsearch, Logstash and Kibana stack lets you search and analyze your logs, utilization and network activity data with ease。
Ready-to-go Docker configuration described in my other project。

Distributed tracing

Analyzing problems in distributed systems can be difficult, for example, tracing requests that propagate from one microservice to another。 It can be quite a challenge to try to find out how a request travels through the system, especially if you don't have any insight into the implementation of a microservice。 Even when there is logging, it is hard to tell which action correlates to a single request。

Spring Cloud Sleuth solves this problem by providing support for distributed tracing。 It adds two types of IDs to the logging: traceId and spanId。 The spanId represents a basic unit of work, for example sending an HTTP request。 The traceId contains a set of spans forming a tree-like structure。 For example, with a distributed big-data store, a trace might be formed by a PUT request。 Using traceId and spanId for each operation we know when and where our application is as it processes a request, making reading our logs much easier。

The logs are as follows, notice the [appname,traceId,spanId,exportable] entries from the Slf4J MDC:

2018-07-26 23:13:49。381  WARN [gateway,3216d0de1384bb4f,3216d0de1384bb4f,false] 2999 --- [nio-4000-exec-1] o。s。c。n。z。f。r。s。AbstractRibbonCommand    : The Hystrix timeout of 20000ms for the command account-service is set lower than the combination of the Ribbon read and connect timeout, 80000ms。
2018-07-26 23:13:49。562  INFO [account-service,3216d0de1384bb4f,404ff09c5cf91d2e,false] 3079 --- [nio-6000-exec-1] c。p。account。service。AccountServiceImpl   : new account has been created: test

• appname: The name of the application that logged the span from the property spring。application。name
• traceId: This is an ID that is assigned to a single request, job, or action
• spanId: The ID of a specific operation that took place
• exportable: Whether the log should be exported to Zipkin

Security

An advanced security configuration is beyond the scope of this proof-of-concept project。 For a more realistic simulation of a real system, consider to use https, JCE keystore to encrypt Microservices passwords and Config server properties content (see documentation for details)。

Infrastructure automation

Deploying microservices, with their interdependence, is much more complex process than deploying monolithic application。 It is important to have fully automated infrastructure。 We can achieve following benefits with Continuous Delivery approach:

• The ability to release software anytime
• Any build could end up being a release
• Build artifacts once - deploy as needed

Here is a simple Continuous Delivery workflow, implemented in this project:

<img width="880" src="https://cloud。githubusercontent。com/assets/6069066/14159789/0dd7a7ce-f6e9-11e5-9fbb-a7fe0f4431e3。png">

In this configuration, Travis CI builds tagged images for each successful git push。 So, there are always latest image for each microservice on Docker Hub and older images, tagged with git commit hash。 It's easy to deploy any of them and quickly rollback, if needed。

How to run all the things?

Keep in mind, that you are going to start 8 Spring Boot applications, 4 MongoDB instances and RabbitMq。 Make sure you have 4 Gb RAM available on your machine。 You can always run just vital services though: Gateway, Registry, Config, Auth Service and Account Service。

Before you start

• Install Docker and Docker Compose。
• Export environment variables: CONFIG_SERVICE_PASSWORD, NOTIFICATION_SERVICE_PASSWORD, STATISTICS_SERVICE_PASSWORD, ACCOUNT_SERVICE_PASSWORD, MONGODB_PASSWORD (make sure they were exported: printenv)
• Make sure to build the project: mvn package [-DskipTests]

Production mode

In this mode, all latest images will be pulled from Docker Hub。
Just copy docker-compose。yml and hit docker-compose up

Development mode

If you'd like to build images yourself (with some changes in the code, for example), you have to clone all repository and build artifacts with maven。 Then, run docker-compose -f docker-compose。yml -f docker-compose。dev。yml up

docker-compose。dev。yml inherits docker-compose。yml with additional possibility to build images locally and expose all containers ports for convenient development。

Important endpoints

• http://localhost:80 - Gateway
• http://localhost:8761 - Eureka Dashboard
• http://localhost:9000/hystrix - Hystrix Dashboard (Turbine stream link: http://turbine-stream-service:8080/turbine/turbine。stream)
• http://localhost:15672 - RabbitMq management (default login/password: guest/guest)

Notes

All Spring Boot applications require already running Config Server for startup。 But we can start all containers simultaneously because of depends_on docker-compose option。

Also, Service Discovery mechanism needs some time after all applications startup。 Any service is not available for discovery by clients until the instance, the Eureka server and the client all have the same metadata in their local cache, so it could take 3 heartbeats。 Default heartbeat period is 30 seconds。

Contributions are welcome!

PiggyMetrics is open source, and would greatly appreciate your help。 Feel free to suggest and implement improvements。