2. sharding-jdbc源码之Configuration

作者: 阿飞的博客 | 来源:发表于2018-01-26 11:19 被阅读775次

阿飞Javaer，转载请注明原创出处，谢谢！

上篇文章sharding-jdbc源码之数据源介绍了通过Java硬编码创建ShardingDataSource。这篇文章通过分析sharding-jdbc-config-parent模块，学习如何通过YAML配置或者spring配置创建ShardingDataSource；sharding-jdbc-config-parent模块包含了三个子模块，关系如下图所示：
sharding-jdbc-config-parent
|__sharding-jdbc-config-common
|__sharding-jdbc-config-spring
|__sharding-jdbc-config-yaml

无论是yaml方式还是spring方式配置ShardingDataSource，最终都会转化为sharding-jdbc-config-common中定义的对象；接下来对两种方式进行源码分析：

YAML配置

可以通过sharding-jdbc-example-config-yaml模块中YamlWithAssignedDataSourceMain.java进行debug；通过YamlWithAssignedDataSourceMain.java源码可知，yaml方式配置数据库的核心源码在YamlShardingDataSource中；

public final class YamlWithAssignedDataSourceMain {
    
    public static void main(final String[] args) throws Exception {
        YamlShardingDataSource dataSource =  new YamlShardingDataSource(
            new File(YamlWithAssignedDataSourceMain.class.getResource("/META-INF/withAssignedDataSource.yaml").getFile()));
        ... ...
    }
}

说明：withAssignedDataSource.yaml的内容请自行查看源码；

com.dangdang.ddframe.rdb.sharding.config.yaml.api.YamlShardingDataSource.java位于sharding-jdbc-config-yaml模块中，核心源码如下：

public class YamlShardingDataSource extends ShardingDataSource {
    
    // 通过yaml文件配置数据源的方式
    public YamlShardingDataSource(final File yamlFile) throws IOException, SQLException {
        // unmarshal(yamlFile)方法是解析yaml文件的核心源码，其作用是将yaml文件解释为YamlConfig（父类是ShardingRuleConfig）
        super(new ShardingRuleBuilder(yamlFile.getName(), unmarshal(yamlFile)).build(), unmarshal(yamlFile).getProps());
    }
    
    ... ...

    private static YamlConfig unmarshal(final File yamlFile) throws IOException {
        try (
                FileInputStream fileInputStream = new FileInputStream(yamlFile);
                InputStreamReader inputStreamReader = new InputStreamReader(fileInputStream, "UTF-8")
        ) {
            // yaml解释依赖第三方组件：org.yaml.snakeyaml; config-all.yaml内容解释成ShardingRuleConfig
            return new Yaml(new Constructor(YamlConfig.class)).loadAs(inputStreamReader, YamlConfig.class);
        }
    }

    private static YamlConfig unmarshal(final byte[] yamlByteArray) throws IOException {
        return new Yaml(new Constructor(YamlConfig.class)).loadAs(new ByteArrayInputStream(yamlByteArray), YamlConfig.class);

    }
}

通过这段源码可知，接下来就会调用ShardingDataSource的构造方法，因为YamlShardingDataSource构造方法中调用了super()，而且YamlShardingDataSource继承自ShardingDataSource；

spring配置

可以通过sharding-jdbc-example-config-spring模块中SpringNamespaceWithAssignedDataSourceMain.java进行debug；其源码就是加载applicationContextWithAssignedDataSource.xml文件，该文件中<rdb>节点即sharding-jdbc定义节点部分的内容如下：

<rdb:strategy id="databaseStrategy" sharding-columns="user_id" algorithm-expression="dbtbl_${user_id.longValue() % 2}"/>
<rdb:strategy id="orderTableStrategy" sharding-columns="order_id" algorithm-expression="t_order_${order_id.longValue() % 4}"/>
<rdb:strategy id="orderItemTableStrategy" sharding-columns="order_id" algorithm-class="com.dangdang.ddframe.rdb.sharding.example.config.spring.algorithm.SingleKeyModuloTableShardingAlgorithm"/>
<rdb:data-source id="shardingDataSource">
    <rdb:sharding-rule data-sources="dbtbl_0,dbtbl_1,dbtbl_config">
        <rdb:table-rules>
            <rdb:table-rule logic-table="t_config" actual-tables="dbtbl_config.t_config"/>
            <rdb:table-rule logic-table="t_order" actual-tables="dbtbl_${0..1}.t_order_${0..3}" 
            database-strategy="databaseStrategy" table-strategy="orderTableStrategy"/>
            <rdb:table-rule logic-table="t_order_item" actual-tables="
            dbtbl_${0..1}.t_order_item_0,
            dbtbl_${0..1}.t_order_item_1,
            dbtbl_${0..1}.t_order_item_2,
            dbtbl_${0..1}.t_order_item_3" 
            database-strategy="databaseStrategy" table-strategy="orderItemTableStrategy"/>
        </rdb:table-rules>
        <rdb:default-database-strategy sharding-columns="none" algorithm-class="com.dangdang.ddframe.rdb.sharding.api.strategy.database.NoneDatabaseShardingAlgorithm"/>
        <rdb:default-table-strategy sharding-columns="none" algorithm-class="com.dangdang.ddframe.rdb.sharding.api.strategy.table.NoneTableShardingAlgorithm"/>
    </rdb:sharding-rule>
</rdb:data-source>

配置文件基于inline表达式，部分内容解读如下：

逻辑表表名为t_order，其实际表是dbtbl_${0..1}.t_order_${0..3}；

t_order表的分表策略，通过table-strategy指定，即orderTableStrategy--根据order_id列的值对4取模；

t_order_item分表策略，通过table-strategy指定，即orderItemTableStrategy--实现算法就是根据根据order_id列对4取模；具体实现请参考SingleKeyModuloTableShardingAlgorithm；

数据库的分库策略，通过database-strategy指定，即databaseStrategy--根据user_id列的值对2取模；

默认数据库和表的分库分表策略：不需要根据任何列水平切分（sharding-columns="none"）；

通过sharding-jdbc-config-spring模块中spring.handlers里的配置http\://www.dangdang.com/schema/ddframe/rdb=com.dangdang.ddframe.rdb.sharding.spring.namespace.handler.ShardingJdbcNamespaceHandler
可知，spring.xml中的<rdb>节点由ShardingJdbcNamespaceHandler进行解析，核心源码如下：

public final class ShardingJdbcNamespaceHandler extends NamespaceHandlerSupport {
    
    @Override
    public void init() {
        // 注册<rdb:strategy>节点的解析器为ShardingJdbcStrategyBeanDefinitionParser
        registerBeanDefinitionParser("strategy", new ShardingJdbcStrategyBeanDefinitionParser());
        // 注册<rdb:data-source>节点的解析器为ShardingJdbcDataSourceBeanDefinitionParser
        registerBeanDefinitionParser("data-source", new ShardingJdbcDataSourceBeanDefinitionParser());
        // 注册<rdb:master-slave-data-source>节点的解析器为MasterSlaveDataSourceBeanDefinitionParser
        registerBeanDefinitionParser("master-slave-data-source", new MasterSlaveDataSourceBeanDefinitionParser());
    }
}

spring.xml中data-source节点剖析：
根据上面ShardingJdbcNamespaceHandler里的源码可知，<rdb:data-source>节点由ShardingJdbcDataSourceBeanDefinitionParser解析，核心源码如下；

// 自定义Parser一定要实现org.springframework.beans.factory.xml.AbstractBeanDefinitionParser才能作为spring.xml中节点中的解析器，这是spring的约定；
public class ShardingJdbcDataSourceBeanDefinitionParser extends AbstractBeanDefinitionParser {
    
    @Override
    // 这是解析入口，这时element是spring.xml中`<rdb:data-source>`节点；
    protected AbstractBeanDefinition parseInternal(final Element element, final ParserContext parserContext) {
        // 准备把`<rdb:data-source>`节点中数据解析成为SpringShardingDataSource
        BeanDefinitionBuilder factory = BeanDefinitionBuilder.rootBeanDefinition(SpringShardingDataSource.class);
        // 解析成SpringShardingDataSource且增加两个构造方法中属性的值（由后面SpringShardingDataSource.java定义可知，构造方法需要两个参数：一个是ShardingRuleConfig类型，一个是Properties类型）
        factory.addConstructorArgValue(parseShardingRuleConfig(element, parserContext));
        factory.addConstructorArgValue(parseProperties(element, parserContext));
        factory.setDestroyMethodName("close");
        return factory.getBeanDefinition();
    }
    
    // 这是解析SpringShardingDataSource构造方法中ShardingRuleConfig类型参数的值
    private BeanDefinition parseShardingRuleConfig(final Element element, final ParserContext parserContext) {
        // 先获取<rdb:sharding-rule>节点
        Element shardingRuleElement = DomUtils.getChildElementByTagName(element, ShardingJdbcDataSourceBeanDefinitionParserTag.SHARDING_RULE_CONFIG_TAG);
        // 将这个节点内容解析成ShardingRuleConfig（参数后面的ShardingRuleConfig.java定义）
        BeanDefinitionBuilder factory = BeanDefinitionBuilder.rootBeanDefinition(ShardingRuleConfig.class);
        // ShardingRuleConfig中dataSource属性赋值
        factory.addPropertyValue("dataSource", parseDataSources(shardingRuleElement, parserContext));
        // ShardingRuleConfig中defaultDataSourceName属性赋值
        parseDefaultDataSource(factory, shardingRuleElement);
        // ShardingRuleConfig中tables属性赋值
        factory.addPropertyValue("tables", parseTableRulesConfig(shardingRuleElement));
        // ShardingRuleConfig中bindingTables属性赋值
        factory.addPropertyValue("bindingTables", parseBindingTablesConfig(shardingRuleElement));
        // ShardingRuleConfig中defaultDatabaseStrategy属性赋值
        factory.addPropertyValue("defaultDatabaseStrategy", parseDefaultDatabaseStrategyConfig(shardingRuleElement));
        // ShardingRuleConfig中defaultTableStrategy属性赋值
        factory.addPropertyValue("defaultTableStrategy", parseDefaultTableStrategyConfig(shardingRuleElement));
        // ShardingRuleConfig中keyGeneratorClass属性赋值
        parseKeyGenerator(factory, shardingRuleElement);
        return factory.getBeanDefinition();
    }

SpringShardingDataSource.java定义：

public class SpringShardingDataSource extends ShardingDataSource {
    // parseInternal()中解析完spring.xml中的<rdb:data-source>节点后，调用这个构造方法
    public SpringShardingDataSource(final ShardingRuleConfig shardingRuleConfig, final Properties props) throws SQLException {
        super(new ShardingRuleBuilder(shardingRuleConfig).build(), props);
    }
}

ShardingDataSource剖析

无论是yaml配置还是spring.xml配置，最终都会调用ShardingDataSource里的构造方法，接下来对其进行分析；

public class ShardingDataSource extends AbstractDataSourceAdapter implements AutoCloseable {
    public ShardingDataSource(final ShardingRule shardingRule, final Properties props) throws SQLException {
        super(shardingRule.getDataSourceRule().getDataSources());
        shardingProperties = new ShardingProperties(null == props ? new Properties() : props);
        // 默认值是CPU核心数
        int executorSize = shardingProperties.getValue(ShardingPropertiesConstant.EXECUTOR_SIZE);
        // ExecutorEngine的构造依赖于google-guava的MoreExecutors
        executorEngine = new ExecutorEngine(executorSize);
        // 是否有配置文件配置了sql_show
        boolean showSQL = shardingProperties.getValue(ShardingPropertiesConstant.SQL_SHOW);
        shardingContext = new ShardingContext(shardingRule, getDatabaseType(), executorEngine, showSQL);
    }
    ...
}

通过该构造方法的源码可知：申明的数据源集合，例如spring.xml中<rdb:sharding-rule data-sources="dbtbl_0,dbtbl_1,dbtbl_config"> ，所有数据源必须是相同的数据库类型；要么全是MySQL，要么全是Oracle；否则抛出异常：Database type inconsistent with '%s' and '%s'；其数据库类型根据connection.getMetaData().getDatabaseProductName()得到；

另外，通过这段源码可知，可配置的属性有sql_show和executor.size，定义在ShardingPropertiesConstant.java中：

两个属性在spring.xml中的配置参考：

<rdb:data-source id="shardingDataSource">
    <rdb:sharding-rule  data-sources="dbtbl_0,dbtbl_1,dbtbl_config">
        ... ...
    </rdb:sharding-rule>
    <rdb:props>
        <prop key="sql.show">true</prop>
        <prop key="executor.size">2</prop>
    </rdb:props>
</rdb:data-source>

两个属性在yaml文件中的配置参考：

props:
  sql.show: false
  executor.size: 4

附ShardingRuleConfig.java定义：

@Getter
@Setter
public class ShardingRuleConfig {
    
    private Map<String, DataSource> dataSource = new HashMap<>();
    
    private String defaultDataSourceName;
    
    private Map<String, TableRuleConfig> tables = new HashMap<>();
    
    private List<BindingTableRuleConfig> bindingTables = new ArrayList<>();
    
    private StrategyConfig defaultDatabaseStrategy;
    
    private StrategyConfig defaultTableStrategy;
    
    private String keyGeneratorClass;
}

Debug

以spring配置数据源的方式进行debug，Main方法为SpringNamespaceWithAssignedDataSourceMain.java，debug之前，需要执行sharding-jdbc-example-config-spring模块中的all_schema.sql脚本；

YAML解析&lombok实战

通过上面对sharding-jdbc源码的分析，发现sharding-jdbc支持yaml格式配置，且大量使用lombok简化源码，接下来简单实践yaml格式文件如何解析，以及lombok如何使用；

假设需要解析的yaml文件内容如下：

rdb:
  oracle:
    username: OracleUse&1
    password: OrcUse*&1
    driverClassName: oracle.jdbc.OracleDriver
    url: jdbc:oracle:thin:@192.168.0.2:1521:xe
  mysql:
    username: MySQLUse&1
    password: MyUse*&1
    driverClassName: com.mysql.jdbc.Driver
    url: jdbc:mysql://192.168.0.1:3306/financials_rules?autoCommit=true

nosql:
  mongodb:
    username: MongoUse&1
    password: MgoUse*&1
  redis:
    password: RdsUse*&1

newsql:

解析yaml文件的核心代码如下：

public class DataSourceTest {

    /**
     * 这个yaml文件要放在resources目录下
     */
    private static final String YAML_FILE_PATH = "datasource.yaml";

    public static void main(String[] args) throws Exception {
        System.out.println(JSON.toJSONString(
                unmarshal(DataSourceTest.class.getClassLoader().getResourceAsStream(YAML_FILE_PATH))));
    }

    private static DataSourceConfig unmarshal(final InputStream is) throws IOException {
        try (
                InputStreamReader inputStreamReader = new InputStreamReader(is, "UTF-8")
        ) {
            return new Yaml(new Constructor(DataSourceConfig.class)).loadAs(inputStreamReader, DataSourceConfig.class);
        }
    }

}

DataSourceConfig.java源码如下：

@Getter
@Setter
public class DataSourceConfig {

    private Map<String, DataSourceItemConfig> rdb;

    private Map<String, DataSourceItemConfig> nosql;

    private Map<String, DataSourceItemConfig> newsql;

}

DataSourceItemConfig.java源码如下：

@Getter
@Setter
public class DataSourceItemConfig {

    private String username;

    private String password;

    private String driverClassName;

    private String url;

}

最终输出结果为：

{
    "nosql": {
        "mongodb": {
            "password": "MgoUse*&1",
            "username": "MongoUse&1"
        },
        "redis": {
            "password": "RdsUse*&1"
        }
    },
    "rdb": {
        "oracle": {
            "driverClassName": "oracle.jdbc.OracleDriver",
            "password": "OrcUse*&1",
            "url": "jdbc:oracle:thin:@192.168.0.2:1521:xe",
            "username": "OracleUse&1"
        },
        "mysql": {
            "driverClassName": "com.mysql.jdbc.Driver",
            "password": "MyUse*&1",
            "url": "jdbc:mysql://192.168.0.1:3306/financials_rules?autoCommit=true",
            "username": "MySQLUse&1"
        }
    }
}

YAML&lombok Maven坐标

<dependency>
    <groupId>org.yaml</groupId>
    <artifactId>snakeyaml</artifactId>
    <version>1.16</version>
</dependency>

<dependency>
    <groupId>org.projectlombok</groupId>
    <artifactId>lombok</artifactId>
    <version>1.16.4</version>
    <scope>provided</scope>
</dependency>

2. sharding-jdbc源码之Configuration

YAML配置

spring配置

ShardingDataSource剖析

Debug

YAML解析&lombok实战

YAML&lombok Maven坐标

相关文章

网友评论

延伸阅读

深度阅读

栏目导航

热点阅读