微服务从零开始之留言板

作者: 老瓦在霸都 | 来源:发表于2016-11-06 22:00 被阅读135次

概述
需求分析
领域对象设计
API 设计
Create Message
Request
Response
Retrieve Message
Update Message
Delete Message
Query Message
TDD - Test Driven Development
测试方法 Test methods
测试矩阵 Test Matrix
留言板实现细节
搭建骨架
构建工具和插件
我们需要哪些库
日志库
测试库
框架及工具库
度量相关库
基本骨架
web.xml
数据对象
配置类
数据库创建
MessageDb
*留言板的主要实现
MessageContoller
MessageService
MessageDao
参考

概述

以一个最简单的留言本为例，麻雀虽小，五脏俱全, 它基本上牵涉到了一个微服务的各个方面, 让我们看看如何从零开始，从无到有构建一个微服务

需求分析

让我们先从用户故事开始, 用例比较多, 可以分优先级分步实施

留言本看下来简单, 其实牵涉到 Web 开发的各个方面, 类似一个小微博

User story	Priority
注册	B
登录	B
用户管理	B
访客留言	A
用户及访客评论	B
留言管理	C
评论管理	C

优先级可以参考时间管理的任务重要性划分

A 重要且紧急
B 重要不紧急
C 紧急不重要
D 不紧急不重要

Paste_Image.png

用户故事图脚本

[Guest]-(Post Message), 
[Guest]-(Query Message), 
[Guest]-(Sign Up), 
[User]-(Sign In), 
[User]-(Add Comments), 
[User]-(Update Message), 
[User]-(Query Message), 
[User]-(Update Self), 
[Admin]-(User Manage), 
[Admin]-(Manage Message), 
(Manage Message)>(Delete Message), 
(Manage Message)>(Delete Comment), 
(Manage Message)>(Archive Message), 
(User Manage)>(Approve Sign Up), 
(User Manage)>(CRUDQ User), 
(CRUDQ User)>(Reset Password), 
(CRUDQ User)>(Lock User)

领域对象设计

对于 guestbook 的最高优先级的用户故事是留言, 也就是 Post Message

让我们先从领域对象设计开始, 留言本的核心对象是 Message 和 Guest

Paste_Image.png

类图脚本如下

%2F%2F Cool Class Diagram, , [Message|id:String;title:String;content:String;tags:String;author: Author; createTime: timestamp], [Message]<>-[Author|id:String;name:String;email: Email;phoneNumber: PhoneNumber;createTime: timestamp]

对应于领域对象如下

消息 Message

attribute	type
id	String(UUID)
title	String
content	String
author	Author
tags	String
createTime	Timestamp

�留言者 Author

attribute	type
id	String(uuid)
name	String
email	String
phoneNumber	String
createTime	Timestamp

API 设计

对象的基本操作是典型的 CRUDQ, 即创建 Create, 获取 Retrieve, 修改 Update, 删除 Delete 和查询 Query

Operation	API
Create Message	POST /messages
Retrieve Message	GET/messages/$id
Update Message	PUT /messages/$id
Delete Message	DELETE/messages/$id
Query Message	GET /messages?$parameters

Create Message

POST /api/v1/messages

Request

{
"�title" : "String",
"content": "String",
"author": {
    "name" : "String",
    "email" : "Email",
    "phoneNumber": "PhoneNumber"
},
"tags" :  "String",
}

Response

{
"url": "http://guestbook.com/api/vi/messages/$id"
"title" : "String",
"content": "String",
"author": {
    "name" : "String",
    "email" : "Email",
    "phoneNumber": "PhoneNumber"
},
"tags" :  "String",
}

Retrieve Message

GET /api/v1/messages/:id

Update Message

PUT /api/v1/messages/:id

Delete Message

DELETE /api/v1/messages/:id

Query Message

GET /api/v1/messages

| Parameter | Type | Default | Comments |
|:----------|------|-----------|---------|---------|
| start | integer | 0 | start number |
| limit | integer | 20 | message count |
| order | string | asc | asc or desc |
| sortby | string | title | id, title, author, email, createtime |
| field | string | * | title, content, author name or email |
| keyword | string | n/a | |

TDD - Test Driven Development

测试驱动开发已经深入人心, 从下到上, 从单元测试到集成测试, 这些是质量的保证
一般来说, 测试的行覆盖率起码要在 80% 以上

测试金字塔我们都有所耳闻，case多了，速度慢了，逻辑越复杂，测试越脆弱，测试集的归类，统计很重要

Paste_Image.png

测试方法 Test methods

单元测试 Unit Test: TestNG, Mockito, SpringTesting
接收测试 API Test: HttpClient
端到端集成测试 E2E Test: Selenium

测试矩阵 Test Matrix

Test Case	Category	Comments
消息创建 Message:Create	UT,API
消息修改 Message:Update	UT,API,E2E
消息删除 Message:Delete	UT, API
消息简单查询 Message:Retrieve	UT
消息复杂查询 Message:Query	UT, API	分页, 排序,根据关键字查询

好了, 到此为止, 咱们已经搞清楚需求和领域对象了, 可以动手开始编程了

且慢, 想想这是不是就够了, 做了就要上线, 上线之后我们最关心什么

关注点	度量
功能是否正常完备	Function Metrics
用量如何	Usage Metrics
性能如何	Performance Metrics
有无异常	Exception Metrics
有无遭受攻击	Fraud attack Metrics
出现问题的修复时间	Fileover/Recover Metrics

让我们在编码实现的时候, 把这些记在心头.

留言板实现细节

以大家比较熟悉的 Java Web App 为例

技术选型如下

前端框架: AngularJS
后端框架: SpringMVC
数据库: SQLite

搭建骨架

mvn archetype:generate -DgroupId=com.github.walterfan -DartifactId=guestbook  -DarchetypeArtifactId=maven-archetype-webapp -DinteractiveMode=false

这样 maven 就为你创建了一个 Java Web App 的骨架

guestbook//pom.xml
guestbook//src
guestbook//src/main
guestbook//src/main/resources
guestbook//src/main/webapp
guestbook//src/main/webapp/index.jsp
guestbook//src/main/webapp/WEB-INF
guestbook//src/main/webapp/WEB-INF/web.xml

Java Web App 的开发框架汗牛充栋, 比如 Struts2, Spring MVC, 还有最近比较流行的 DropWizard 和 Spring Boot

这两个框架都是众多开源项目的集大成者, 先不用这么重的东西来做留言本
这里就用Spring Boot 的核心项目 Spring MVC 来快速实现

具体实现下节细说, 这里讲几句题外话, 很多Java 开发者都有"好读书, 不求甚解"的坏毛病, 包括我在内, 从 C/C++ 世界转过来, 发现 Java 太方便了, 开发效率极高, 各种库和框架让人目不暇接, 很容易就迷失了

有时间还是可以看一看 HTTP 协议和 Servlet JSR(Java Specification Requests)
最近一版是 JSR 340: Java Servlet 3.1 Specification

归根到底, 它是一个网络应用程序, 程序启动时侦听 80 或其他端口, 接收 HTTP Request, 解包交应用逻辑进行一些处理后以 HTTP Response 的编码返回.

只不过，现在通过Servlet 容器把这些底层的脏活累活干了，交到 Application 手里已经是标准的 HttpRequest 和 HttpResponse

构建工具和插件

Maven 是Java世界的标配, 近年来gradel 异军突起, 有待时间的检验

参见详情: https://github.com/walterfan/guestbook/blob/master/pom.xml

Maven 的插件也是林林总总, 不胜枚举, 这里只选用一些常用的

maven-surefire-plugin for uni test
Jacoco-maven-plugin for test coverage
maven-failsafe-plugin for integration test

我们需要哪些库

日志库

sl4j
logback

测试库

testng
mockito
spring test

框架及工具库

Spring MVC
Jackson
guava
commons lang3, io,

度量相关库

Metrics
InfluxDB

基本骨架

Spring MVC 原理回顾, DispatchServlet 是其核心

Controller
Service
Domain
Dao
Metrics: 度量相关代码

web.xml

<?xml version="1.0" ?>
<web-app xmlns="http://java.sun.com/xml/ns/javaee"
         xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
         xsi:schemaLocation="http://java.sun.com/xml/ns/javaee http://java.sun.com/xml/ns/javaee/web-app_3_0.xsd"
         version="3.0">

<description>Micro Service</description>

<context-param>
  <param-name>contextClass</param-name>
  <param-value>org.springframework.web.context.support.AnnotationConfigWebApplicationContext</param-value>
</context-param>

<context-param>
  <param-name>contextConfigLocation</param-name>
  <param-value>com.github.walterfan.guestbook.MessageConfig</param-value>
</context-param>

<listener>
  <listener-class>org.springframework.web.context.ContextLoaderListener</listener-class>
</listener>

<servlet>
  <servlet-name>mvc-servlet</servlet-name>
  <servlet-class>org.springframework.web.servlet.DispatcherServlet</servlet-class>
  <init-param>
    <param-name>contextConfigLocation</param-name>
    <param-value></param-value>
  </init-param>
  <load-on-startup>1</load-on-startup>
</servlet>

<servlet-mapping>
  <servlet-name>mvc-servlet</servlet-name>
  <url-pattern>/*</url-pattern>
</servlet-mapping>

</web-app>

数据对象

1. class Message

package com.github.walterfan.guestbook.domain;


import org.hibernate.validator.constraints.NotBlank;

import javax.validation.constraints.NotNull;
import java.util.Date;


public class Message extends BaseObject {
    private String id;
    @NotBlank
    private String title;
    @NotBlank
    private String content;
    @NotNull
    private Author author;
    private String tags;
    private Date createTime;

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public String getTitle() {
        return title;
    }

    public void setTitle(String title) {
        this.title = title;
    }

    public String getContent() {
        return content;
    }

    public void setContent(String content) {
        this.content = content;
    }

    public Author getAuthor() {
        return author;
    }

    public void setAuthor(Author author) {
        this.author = author;
    }

    public String getTags() {
        return tags;
    }

    public void setTags(String tags) {
        this.tags = tags;
    }

    public Date getCreateTime() {
        return createTime;
    }

    public void setCreateTime(Date createTime) {
        this.createTime = createTime;
    }
}

配置类

代替之前的spring bean xml 配置文件
1. class MessageConfig

    
package com.github.walterfan.guestbook;

import com.github.walterfan.guestbook.controller.IndexController;
import com.github.walterfan.guestbook.controller.MessageController;
import com.github.walterfan.guestbook.dao.MessageDao;
import com.github.walterfan.guestbook.dao.MessageMapper;
import com.github.walterfan.guestbook.service.MessageService;
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.SqlSessionFactoryBean;
import org.mybatis.spring.SqlSessionTemplate;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Import;
import org.springframework.core.env.Environment;
import org.springframework.jdbc.datasource.DataSourceTransactionManager;
import org.springframework.jdbc.datasource.SimpleDriverDataSource;
import org.springframework.web.servlet.config.annotation.EnableWebMvc;

import javax.sql.DataSource;
import java.sql.Driver;

/**
 * Created by walter on 06/11/2016.
 */
@Configuration
@EnableWebMvc
@Import({
        IndexController.class,
        MessageController.class
})
public class MessageConfig {

    @Autowired
    private Environment env;

    @Bean
    public MessageService messageService() {
        return new MessageService();
    }

    @Bean
    public MessageProperties messageProperties()  {
        return new MessageProperties();
    }


    @Bean
    public DataSource dataSource() throws ClassNotFoundException {
        SimpleDriverDataSource dataSource = new SimpleDriverDataSource();
        dataSource.setDriverClass((Class<? extends Driver>)   Class.forName(messageProperties().getJdbcDriver()));
        dataSource.setUsername(messageProperties().getJdbcUserName());
        dataSource.setUrl(messageProperties().getJdbcUrl());
        dataSource.setPassword(messageProperties().getJdbcPassword());

        return dataSource;
    }

    @Bean
    public DataSourceTransactionManager transactionManager() throws ClassNotFoundException {
        return new DataSourceTransactionManager(dataSource());
    }


    @Bean
    public SqlSessionFactory sqlSessionFactory() throws Exception {
        SqlSessionFactoryBean sqlSessionFactory = new SqlSessionFactoryBean();
        sqlSessionFactory.setDataSource(dataSource());
        return (SqlSessionFactory) sqlSessionFactory.getObject();
    }

    @Bean
    public MessageDao messageDao() throws Exception {
        SqlSessionFactory sessionFactory = sqlSessionFactory();
        sessionFactory.getConfiguration().addMapper(MessageMapper.class);

        SqlSessionTemplate sessionTemplate = new SqlSessionTemplate(sqlSessionFactory());
        return sessionTemplate.getMapper(MessageMapper.class);
    }
}

数据库创建

我们可以用反射的方法直接生成创建,删除以及插入表数据的语句, 而不必自己手写SQL, Hibernate框架也用了类似的方法, 这里简单实现一个数据库创建初始化的工具

配置如下 jdbc.properties

jdbc.driverClass=org.sqlite.JDBC
jdbc.url=jdbc:sqlite:/workspace/walter/wfdb.s3db
#jdbc.driverClass=com.mysql.jdbc.Driver
#jdbc.url=jdbc:mysql://localhost/wfdb?useUnicode=true&characterEncoding=utf8
jdbc.username=walter
jdbc.password=pass

MessageDb

package com.github.walterfan.guestbook.db;

import com.github.walterfan.guestbook.domain.Message;

import java.sql.SQLException;
import java.util.Date;
import java.util.UUID;

import static java.lang.System.out;

/**
 * Created by walter on 07/11/2016.
 */
public class MessageDb {

    private final DbConn dbConn;

    private static String CHECK_SQL = "SELECT * FROM sqlite_master WHERE type='table' and name='%s'";

    public MessageDb() throws Exception {
        dbConn = new DbConn("jdbc.properties");
        dbConn.setDebug(true);
        dbConn.createConnection();
    }

    public void init() throws Exception {
        int ret = initTable();
        if(ret > 0) {
            initData();

        }
    }

    public int initTable() throws Exception {
        int ret = check(Message.class);
        if(ret > 0) {
            out.println("found table and drop it firstly ");
            dropTable(Message.class);
        }

        createTable(Message.class);
        return check(Message.class);



    }

    private int initData() throws Exception {
        String id = UUID.randomUUID().toString();
        Message msg = new Message();
        msg.setId(id);
        msg.setTitle("hello guest");
        msg.setContent("this is a test message");
        msg.setTags("test tag");
        msg.setCreateTime(new Date());
        String sql = DbHelper.makeInsertSql(msg);
        out.println("execute " + sql);
        dbConn.execute(sql);

        sql = DbHelper.makeQuerySql(msg.getClass(), String.format("id = '%s'", id));
        out.println("execute " + sql);
        return dbConn.execute(sql);
    }

    public int createTable(Class<?> clazz) throws Exception {

        String sql = DbHelper.makeCreateTableSql(clazz);
        out.println("execute " + sql);
        return dbConn.execute(sql);
    }

    public int dropTable(Class<?> clazz) throws Exception {

        String sql = DbHelper.makeDropTableSql(clazz);
        out.println("execute " + sql);
        return dbConn.execute(sql);
    }

    public void clean() throws SQLException {
        dbConn.commit();
        dbConn.closeConnection();
    }

    public int check(Class<?> clazz) throws Exception {
        String sql = String.format(CHECK_SQL, clazz.getSimpleName().toLowerCase());
        out.println("execute " + sql);
        return dbConn.execute(sql);

    }

    public static void main(String[] argv) throws Exception {
        MessageDb db = new MessageDb();
        db.init();
        db.clean();
    }

 }

其他代码参见 https://github.com/walterfan/guestbook/tree/master/src/main/java/com/github/walterfan/guestbook/db

执行结果如下:

execute SELECT * FROM sqlite_master WHERE type='table' and name='message'

type	name	tbl_name	rootpage	sql
table	message	message	48	CREATE TABLE message (id TEXT,title

found table and drop it firstly
execute DROP TABLE message
execute CREATE TABLE message (id TEXT,title TEXT,content TEXT,author TEXT,tags TEXT,createTime DATETIME)
execute SELECT * FROM sqlite_master WHERE type='table' and name='message'

type	name	tbl_name	rootpage	sql
table	message	message	48	CREATE TABLE message (id TEXT,title

execute insert into message(id,title,content,author,tags,createTime) values('fa2ba0d0-d843-48e1-804d-641647d33b5b','hello guest','this is a test message',null,'test tag','2016-11-17 23:08:34.959')
execute select * from message where id = 'fa2ba0d0-d843-48e1-804d-641647d33b5b'

id	title	content	author	tags	createTime
fa2ba0d0-d843-48e1-804d-641647d33b5b	hello guest	this is a test message		test tag	2016-11-17 23:08:34.959

�留言板的主要实现

按照标准的 SpringMVC 结构, 主要用五个类来搞定, 前面两个前面已经提过了, 所有代码请参见 https://github.com/walterfan/guestbook, 下面是所谓的 CRC( Class Responsibility Collaborator ) 类职责与协作者表格

Class	Responsibility	Collaborator
1. Message	留言数据对象	所有类
2. MessageConfig	留言板配置	MessageController, MessageService, MessageDao
3. MessageController	留言板控制器	MessageService
4. MessageService	留言板服务	MessageDao, MessageController
5. MessageDao	留言板数据存储接口	MessageService
6. MessageMapper	留言板数据存储实现	MessageService

注: 为简单起见, 这里只用了一个数据对象 Message , 也可以细分为

Class	Responsibility	Collaborator
MessageDto	数据传输对象	MessageService, MessageDao
MessageEntity	数据�实体对象	MessageService, MessageDao
MessageBo	数据�业务对象	MessageService
MessageRequest	数据�请求对象	MessageController
MessageResponse	数据�响应对象	MessageController

MessageContoller

3 . class MessageController

package com.github.walterfan.guestbook.controller;

import com.github.walterfan.guestbook.domain.GenericQuery;
import com.github.walterfan.guestbook.domain.Message;
import com.github.walterfan.guestbook.service.MessageService;
import org.apache.commons.lang3.StringUtils;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.web.bind.annotation.PathVariable;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RequestMapping;
import org.springframework.web.bind.annotation.RequestMethod;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;

import javax.validation.Valid;
import java.util.List;

@RestController
@RequestMapping(value = "/guestbook/api/v1/", produces = { "application/json" })
public class MessageController {


    protected final Logger logger = LoggerFactory.getLogger(getClass());

    @Autowired
    private MessageService messageService;


    @RequestMapping(value = "/messages", method = RequestMethod.POST)
    public Message createMessage(@Valid @RequestBody Message message) throws Exception {
        logger.info("got post request: " + message.toString());
        messageService.createMessage(message);
        return message;
    }

    @RequestMapping(value = {"/messages", "/"}, method = RequestMethod.GET)
    public List<Message> queryMessages(@RequestParam(value = "start",   required = false) Integer start,
                                       @RequestParam(value = "limit",   required = false) Integer limit,
                                       @RequestParam(value = "order",   required = false) String order,
                                       @RequestParam(value = "sortBy",  required = false) String sortBy,
                                       @RequestParam(value = "keyword", required = false) String keyword,
                                       @RequestParam(value = "fieldName",   required = false) String fieldName) {
        logger.info("query messages request");

        GenericQuery query = new GenericQuery();
        if(null != start) query.setStart(start);
        if(null != limit) query.setLimit(limit);
        if(null != order) {
            if("ASC".equalsIgnoreCase(order)) {
                query.setOrder(GenericQuery.OrderType.ASC);
            } else if("DESC".equalsIgnoreCase(order)) {
                query.setOrder(GenericQuery.OrderType.DESC);
            }
        }
        if(StringUtils.isNotBlank(sortBy)) query.setSortBy(sortBy);
        if(StringUtils.isNotBlank(fieldName)) query.setFieldName(fieldName);
        if(StringUtils.isNotBlank(keyword)) query.setKeyword(keyword);

        List<Message> messageList = messageService.queryMessage(query);
        return messageList;
    }

    @RequestMapping(value = "messages/{id}", method = RequestMethod.GET)
    public Message getMessage(@PathVariable("id") String id) throws Exception {
        return messageService.retrieveMessage(id);
    }


    @RequestMapping(value = "messages/{id}", method = RequestMethod.PUT)
    public Message updateMessage(@PathVariable("id") String id, @RequestBody Message message) {
        message.setId(id);
        messageService.updateMessage(message);
        return message;
    }

    @RequestMapping(value = "messages/{id}", method = RequestMethod.DELETE)
    public void deleteMessage(@PathVariable("id") String id) {
        messageService.deleteMessage(id);

    }
}

MessageService

4 . class MessageService

package com.github.walterfan.guestbook.service;

import com.github.walterfan.guestbook.dao.MessageDao;
import com.github.walterfan.guestbook.domain.GenericQuery;
import com.github.walterfan.guestbook.domain.Message;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.stereotype.Service;

import java.util.List;
import java.util.UUID;


@Service
public class MessageService {

    @Autowired
    private MessageDao messageDao;

    public void createMessage(Message message) {
        UUID id = UUID.randomUUID();
        message.setId(id.toString());
        messageDao.createMessage(message);
    }

    public Message retrieveMessage(String id) {
        return messageDao.retrieveMessage(id);
    }

    public List<Message> queryMessage(GenericQuery query) {
        return messageDao.queryMessage(query);
    }

    public void updateMessage(Message message) {
        messageDao.updateMessage(message);
    }

    public void deleteMessage(String id) {
        messageDao.deleteMessage(id);

    }
}

MessageDao

5 . class MessageDao

package com.github.walterfan.guestbook.dao;

import com.github.walterfan.guestbook.domain.GenericQuery;
import com.github.walterfan.guestbook.domain.Message;

import java.util.List;

public interface MessageDao {

    void createMessage(Message message);


    Message retrieveMessage(String id);

    void updateMessage(Message message);


    void deleteMessage(String id);


    List<Message> queryMessage(GenericQuery query);
}

6 . MessageMapper

package com.github.walterfan.guestbook.dao;

import com.github.walterfan.guestbook.domain.GenericQuery;
import com.github.walterfan.guestbook.domain.Message;
import org.apache.ibatis.annotations.Delete;
import org.apache.ibatis.annotations.Insert;
import org.apache.ibatis.annotations.Select;
import org.apache.ibatis.annotations.Update;

import java.util.List;

public interface MessageMapper extends MessageDao {

    //#{author.id}
    @Insert("INSERT into message(id,title,content,tags, createTime) " +
            "VALUES(#{id}, #{title}, #{content}, #{tags}, #{createTime})")
    void createMessage(Message message);

    @Select("SELECT * FROM message WHERE id = #{id}")
    Message retrieveMessage(String id);

    @Update("UPDATE message SET title=#{title}, content =#{content}, tags=#{tags} , " +
            " WHERE id =#{id}")
    void updateMessage(Message message);

    @Delete("DELETE FROM message WHERE id =#{id}")
    void deleteMessage(String id);

    @Select("SELECT * FROM message ")
    List<Message> queryMessage(GenericQuery query);

}

至此, 一个最小的留言板微服务雏形已成, 可以快速看一下效果

mvn jetty:run

Paste_Image.png

好, 就此打住.

好的开始是成功的一半, 虽然我们只完成了整个项目的第一步, 也等于成功了一半.
之后, 让我们一步一步来分析和实现更多功能性和非功能性的需求吧.
即使这么小的一个留言板微服务, 也还有不少细节要仔细考虑和优化.

微服务从零开始之留言板

目录

概述

需求分析

领域对象设计

API 设计

Create Message

Request

Response

Retrieve Message

Update Message

Delete Message

Query Message

TDD - Test Driven Development

测试方法 Test methods

测试矩阵 Test Matrix

留言板实现细节

搭建骨架

构建工具和插件

我们需要哪些库

日志库

测试库

框架及工具库

度量相关库

基本骨架

web.xml

数据对象

配置类

数据库创建

MessageDb

�留言板的主要实现

MessageContoller

MessageService

MessageDao

参考

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读

微服务开发实战

编程笔记

互金

微服务