背景
最近在做一个数据隔离的需求，相同的库表结构，根据不同的用户，数据落到不同的库。第一想到的就是多数据源的方案。
以前做过类似的多数据源的切换方案，是在项目启动的时候就知道数据源，提前加载数据源，现在是只有用户过来的时候才去创建数据源，同时后期又增加了不同类型的用户，不需要修改代码，直接使用。
ps:在网上看了文章，有许多把多数据源切换跟动态数据源加载混在一起，让很多人迷茫，在这里说下
多数据源：同一个项目，用到多个数据源，在项目启动的时候就已经创建出来，比如：库存数据库、订单数据。
动态加载：项目启动的时候不知道应该创建那个数据库，只有根据请求的用户信息，动态创建相应的数据源。
技术实现
整体实现流程
image.png
动态数据源实现
-
动态数据源实现整体流程
image.png
技术实现
动态数据源-AbstractRoutingDataSource
源码
/*
* Copyright 2002-2017 the original author or authors.
*
* Licensed under the Apache License, Version 2.0 (the "License");
* you may not use this file except in compliance with the License.
* You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.springframework.jdbc.datasource.lookup;
import java.sql.Connection;
import java.sql.SQLException;
import java.util.HashMap;
import java.util.Map;
import javax.sql.DataSource;
import org.springframework.beans.factory.InitializingBean;
import org.springframework.jdbc.datasource.AbstractDataSource;
import org.springframework.lang.Nullable;
import org.springframework.util.Assert;
/**
* Abstract {@link javax.sql.DataSource} implementation that routes {@link #getConnection()}
* calls to one of various target DataSources based on a lookup key. The latter is usually
* (but not necessarily) determined through some thread-bound transaction context.
*
* @author Juergen Hoeller
* @since 2.0.1
* @see #setTargetDataSources
* @see #setDefaultTargetDataSource
* @see #determineCurrentLookupKey()
*/
public abstract class AbstractRoutingDataSource extends AbstractDataSource implements InitializingBean {
@Nullable
private Map<Object, Object> targetDataSources;
@Nullable
private Object defaultTargetDataSource;
private boolean lenientFallback = true;
private DataSourceLookup dataSourceLookup = new JndiDataSourceLookup();
@Nullable
private Map<Object, DataSource> resolvedDataSources;
@Nullable
private DataSource resolvedDefaultDataSource;
/**
* Specify the map of target DataSources, with the lookup key as key.
* The mapped value can either be a corresponding {@link javax.sql.DataSource}
* instance or a data source name String (to be resolved via a
* {@link #setDataSourceLookup DataSourceLookup}).
* <p>The key can be of arbitrary type; this class implements the
* generic lookup process only. The concrete key representation will
* be handled by {@link #resolveSpecifiedLookupKey(Object)} and
* {@link #determineCurrentLookupKey()}.
*/
public void setTargetDataSources(Map<Object, Object> targetDataSources) {
this.targetDataSources = targetDataSources;
}
/**
* Specify the default target DataSource, if any.
* <p>The mapped value can either be a corresponding {@link javax.sql.DataSource}
* instance or a data source name String (to be resolved via a
* {@link #setDataSourceLookup DataSourceLookup}).
* <p>This DataSource will be used as target if none of the keyed
* {@link #setTargetDataSources targetDataSources} match the
* {@link #determineCurrentLookupKey()} current lookup key.
*/
public void setDefaultTargetDataSource(Object defaultTargetDataSource) {
this.defaultTargetDataSource = defaultTargetDataSource;
}
/**
* Specify whether to apply a lenient fallback to the default DataSource
* if no specific DataSource could be found for the current lookup key.
* <p>Default is "true", accepting lookup keys without a corresponding entry
* in the target DataSource map - simply falling back to the default DataSource
* in that case.
* <p>Switch this flag to "false" if you would prefer the fallback to only apply
* if the lookup key was {@code null}. Lookup keys without a DataSource
* entry will then lead to an IllegalStateException.
* @see #setTargetDataSources
* @see #setDefaultTargetDataSource
* @see #determineCurrentLookupKey()
*/
public void setLenientFallback(boolean lenientFallback) {
this.lenientFallback = lenientFallback;
}
/**
* Set the DataSourceLookup implementation to use for resolving data source
* name Strings in the {@link #setTargetDataSources targetDataSources} map.
* <p>Default is a {@link JndiDataSourceLookup}, allowing the JNDI names
* of application server DataSources to be specified directly.
*/
public void setDataSourceLookup(@Nullable DataSourceLookup dataSourceLookup) {
this.dataSourceLookup = (dataSourceLookup != null ? dataSourceLookup : new JndiDataSourceLookup());
}
@Override
public void afterPropertiesSet() {
if (this.targetDataSources == null) {
throw new IllegalArgumentException("Property 'targetDataSources' is required");
}
this.resolvedDataSources = new HashMap<>(this.targetDataSources.size());
this.targetDataSources.forEach((key, value) -> {
Object lookupKey = resolveSpecifiedLookupKey(key);
DataSource dataSource = resolveSpecifiedDataSource(value);
this.resolvedDataSources.put(lookupKey, dataSource);
});
if (this.defaultTargetDataSource != null) {
this.resolvedDefaultDataSource = resolveSpecifiedDataSource(this.defaultTargetDataSource);
}
}
/**
* Resolve the given lookup key object, as specified in the
* {@link #setTargetDataSources targetDataSources} map, into
* the actual lookup key to be used for matching with the
* {@link #determineCurrentLookupKey() current lookup key}.
* <p>The default implementation simply returns the given key as-is.
* @param lookupKey the lookup key object as specified by the user
* @return the lookup key as needed for matching
*/
protected Object resolveSpecifiedLookupKey(Object lookupKey) {
return lookupKey;
}
/**
* Resolve the specified data source object into a DataSource instance.
* <p>The default implementation handles DataSource instances and data source
* names (to be resolved via a {@link #setDataSourceLookup DataSourceLookup}).
* @param dataSource the data source value object as specified in the
* {@link #setTargetDataSources targetDataSources} map
* @return the resolved DataSource (never {@code null})
* @throws IllegalArgumentException in case of an unsupported value type
*/
protected DataSource resolveSpecifiedDataSource(Object dataSource) throws IllegalArgumentException {
if (dataSource instanceof DataSource) {
return (DataSource) dataSource;
}
else if (dataSource instanceof String) {
return this.dataSourceLookup.getDataSource((String) dataSource);
}
else {
throw new IllegalArgumentException(
"Illegal data source value - only [javax.sql.DataSource] and String supported: " + dataSource);
}
}
@Override
public Connection getConnection() throws SQLException {
return determineTargetDataSource().getConnection();
}
@Override
public Connection getConnection(String username, String password) throws SQLException {
return determineTargetDataSource().getConnection(username, password);
}
@Override
@SuppressWarnings("unchecked")
public <T> T unwrap(Class<T> iface) throws SQLException {
if (iface.isInstance(this)) {
return (T) this;
}
return determineTargetDataSource().unwrap(iface);
}
@Override
public boolean isWrapperFor(Class<?> iface) throws SQLException {
return (iface.isInstance(this) || determineTargetDataSource().isWrapperFor(iface));
}
/**
* Retrieve the current target DataSource. Determines the
* {@link #determineCurrentLookupKey() current lookup key}, performs
* a lookup in the {@link #setTargetDataSources targetDataSources} map,
* falls back to the specified
* {@link #setDefaultTargetDataSource default target DataSource} if necessary.
* @see #determineCurrentLookupKey()
*/
protected DataSource determineTargetDataSource() {
Assert.notNull(this.resolvedDataSources, "DataSource router not initialized");
Object lookupKey = determineCurrentLookupKey();
DataSource dataSource = this.resolvedDataSources.get(lookupKey);
if (dataSource == null && (this.lenientFallback || lookupKey == null)) {
dataSource = this.resolvedDefaultDataSource;
}
if (dataSource == null) {
throw new IllegalStateException("Cannot determine target DataSource for lookup key [" + lookupKey + "]");
}
return dataSource;
}
##
/**
* Determine the current lookup key. This will typically be
* implemented to check a thread-bound transaction context.
* <p>Allows for arbitrary keys. The returned key needs
* to match the stored lookup key type, as resolved by the
* {@link #resolveSpecifiedLookupKey} method.
*/
@Nullable
protected abstract Object determineCurrentLookupKey();
}
从源码可以看出主要涉及到的变量
Map<Object, Object> targetDataSources;//外部创建的数据源都放在这个集合下
Object defaultTargetDataSource;//指定的默认数据源
Map<Object, DataSource> resolvedDataSources;//内部使用的数据源 跟目标数据源对应
DataSource resolvedDefaultDataSource;//跟默认数据源对应
他们直接的关系是：targetDataSources 是外部调用，resolvedDataSources是内部使用，当选加载数据源的时候，targetDataSources数据源集合赋值给resolvedDataSources
变量使用的地方
public void afterPropertiesSet() {
if (this.targetDataSources == null) {
throw new IllegalArgumentException("Property 'targetDataSources' is required");
}
//数据源集合赋值 默认数据源默认
this.resolvedDataSources = new HashMap<>(this.targetDataSources.size());
this.targetDataSources.forEach((key, value) -> {
Object lookupKey = resolveSpecifiedLookupKey(key);
DataSource dataSource = resolveSpecifiedDataSource(value);
this.resolvedDataSources.put(lookupKey, dataSource);
});
if (this.defaultTargetDataSource != null) {
this.resolvedDefaultDataSource = resolveSpecifiedDataSource(this.defaultTargetDataSource);
}
}
在多数据源切换中，抽象类AbstractRoutingDataSource里面的抽象方法determineCurrentLookupKey必须实现，切换主要是根据这个方法进行切换，代码如下
protected DataSource determineTargetDataSource() {
Assert.notNull(this.resolvedDataSources, "DataSource router not initialized");
//调用此方法拿到，切换数据源的标记，进行切换
Object lookupKey = determineCurrentLookupKey();
DataSource dataSource = this.resolvedDataSources.get(lookupKey);
//这里说明下，默认数据源必须要设置，当数据源集合获取不到数据源的时候，默认使用默认数据源进行兜底resolvedDefaultDataSource
if (dataSource == null && (this.lenientFallback || lookupKey == null)) {
dataSource = this.resolvedDefaultDataSource;
}
if (dataSource == null) {
throw new IllegalStateException("Cannot determine target DataSource for lookup key [" + lookupKey + "]");
}
return dataSource;
}
/**
* Determine the current lookup key. This will typically be
* implemented to check a thread-bound transaction context.
* <p>Allows for arbitrary keys. The returned key needs
* to match the stored lookup key type, as resolved by the
* {@link #resolveSpecifiedLookupKey} method.
*/
@Nullable
protected abstract Object determineCurrentLookupKey();
上面分析了下源码，到此 我们大概知道怎么去实现动态数据源的加载以及切换
具体实现
- 创建一个全部线程变量，控制数据源的切换 DatabaseContextHolder
public class DatabaseContextHolder {
//线程变量，每一个线程一个值，相互隔离
private static ThreadLocal<String> contextHolder = new ThreadLocal<>();
public static void setDatabaseType(String databaseType){
contextHolder.set(databaseType);
}
public static String getDataBaseType(){
return contextHolder.get();
}
public static void clearDbKey(){
contextHolder.remove();
}
}
- 继承AbstractRoutingDataSource创建动态数据源，DynamicDataSource
public class DynamicDataSource extends AbstractRoutingDataSource {
private static final Logger logger = LoggerFactory.getLogger(DynamicDataSource.class);
private static DynamicDataSource instance;
private static byte[] lock=new byte[0];
//数据源集合，
private static Map<Object,Object> dataSourceMap=new HashMap<Object, Object>();
@Override
public void setTargetDataSources(Map<Object, Object> targetDataSources) {
super.setTargetDataSources(targetDataSources);
dataSourceMap.putAll(targetDataSources);
super.afterPropertiesSet();// 必须添加该句，否则新添加数据源无法识别到
}
public Map<Object, Object> getDataSourceMap() {
return dataSourceMap;
}
public static synchronized DynamicDataSource getInstance(){
if(instance==null){
synchronized (lock){
if(instance==null){
instance=new DynamicDataSource();
}
}
}
return instance;
}
@Override
protected Object determineCurrentLookupKey() {
logger.info("当前使用的数据源：{}",DatabaseContextHolder.getDataBaseType());
return DatabaseContextHolder.getDataBaseType();
}
}
此类解释下，单例模式、dataSourceMap，主要是为了在数据源初始化以后，后续有的数据源进来，能动态加载，不至于重新应用, afterPropertiesSet()这个方法必须调用，否则新增加的数据源不会生效，具体看上面的代码
以上两步，动态数据源就创建好了，可以替代DataSource了，接下来，应该怎么使用呢？
- 创建数据源，DataSourceConfig
@Configuration
public class DataSourceConfig {
@Value("${spring.datasource.master.url}")
private String masterDBUrl;
@Value("${spring.datasource.master.username}")
private String masterDBUser;
@Value("${spring.datasource.master.password}")
private String masterDBPassword;
@Value("${spring.datasource.master.driver-class-name}")
private String masterDBDreiverName;
@Bean
public DynamicDataSource dynamicDataSource() {
DynamicDataSource dynamicDataSource = DynamicDataSource.getInstance();
DruidDataSource oneDataSouce = new DruidDataSource();
oneDataSouce.setDriverClassName("com.mysql.jdbc.Driver");
oneDataSouce.setUrl("jdbc:mysql://127.0.0.1:3306/kb_master?serverTimezone=Hongkong");
oneDataSouce.setUsername("");
oneDataSouce.setPassword("");
DruidDataSource twoDataSource = new DruidDataSource();
twoDataSource.setDriverClassName("com.mysql.jdbc.Driver");
twoDataSource.setUrl("jdbc:mysql://127.0.0.1:3306/kb_master?serverTimezone=Hongkong");
twoDataSource.setUsername("");
twoDataSource.setPassword("");
Map<Object,Object> map = new HashMap<>();
map.put("oneDataSouce", oneDataSouce);
map.put("twoDataSource", twoDataSource);
//添加数据源结合
dynamicDataSource.setTargetDataSources(map);
//必须设置一个默认数据源兜底
dynamicDataSource.setDefaultTargetDataSource(oneDataSouce);
return dynamicDataSource;
}
@Bean
public SqlSessionFactory sqlSessionFactory(
@Qualifier("dynamicDataSource") DataSource dynamicDataSource)
throws Exception {
SqlSessionFactoryBean bean = new SqlSessionFactoryBean();
bean.setDataSource(dynamicDataSource);
bean.setMapperLocations(new PathMatchingResourcePatternResolver()
.getResources("classpath*:mappers/*.xml"));
return bean.getObject();
}
@Bean(name = "sqlSessionTemplate")
public SqlSessionTemplate sqlSessionTemplate(
@Qualifier("sqlSessionFactory") SqlSessionFactory sqlSessionFactory)
throws Exception {
return new SqlSessionTemplate(sqlSessionFactory);
}
@Bean
public MapperScannerConfigurer mapperScannerConfigurer() {
MapperScannerConfigurer mapperScannerConfigurer = new MapperScannerConfigurer();
mapperScannerConfigurer.setSqlSessionFactoryBeanName("sqlSessionFactory");
//跟mybatis映射mapper的时候，特别注意，一定要具体到mapper的包下，不然会导致，在bean会被加载两次，例如：我就遇到service 被加载了两次具体大家可以看下MapperScannerConfigurer
mapperScannerConfigurer.setBasePackage("com.cn.kbtest");
return mapperScannerConfigurer;
}
}
我用的是springboot，如果是其他这配置，写在xml里面就行，具体可以自行百度
上面动态数据源基本上就写完了，下面看怎么使用
- 通过AOP的切面进行拦截，可以对包路径进行拦截，也有对指定的注解进行拦截，然后进行相应的数据库操作
@Aspect
@Order
@Component
public class DataSourceViewAspect {
private static final Logger logger = LoggerFactory.getLogger(DataSourceViewAspect.class);
@Pointcut("@within(secondDataSourceMapper)")
public void pointCut(SecondDataSourceMapper secondDataSourceMapper){}
@Before("pointCut(secondDataSourceMapper)")
public void doBefore(JoinPoint point, SecondDataSourceMapper secondDataSourceMapper){
DatabaseContextHolder.setDatabaseType("view");
logger.info("数据源切换为:{}","view");
}
@After("pointCut(secondDataSourceMapper)")
public void after(SecondDataSourceMapper secondDataSourceMapper){
logger.info("清除数据源标记:{}" ,"view");
DatabaseContextHolder.clearDbKey();
}
}
上面是通过拦截这个注解，类下所有的方法，进行切换，也可以拦截package，具体可以查询aop，里面有个注意点，一定要@before，之前就进行切换，不然出错
这样就做到了动态数据源的切换，这是对项目启动的时候，对存在的数据源加载进行切换，如果后续新增了一个数据源，怎么办？
- 通过拦截器，对外部请求进行拦截，可以从header、cookie等里面获取数据源的标记，进行数据源的创建，然后加载
@Component
public class MyControllerAdvice implements HandlerInterceptor {
@Value("${datasource.global.use-encrypted-password}")
private boolean useEncryptedPassword;
@Value("${datasource.dynamic.url}")
private String url;
@Override
public void afterCompletion(HttpServletRequest request, HttpServletResponse response, Object handler, Exception ex)
throws Exception {
HospitalContextHolder.remove();
}
@Value("${datasource.dynamic.username}")
private String userName;
@Value("${datasource.dynamic.password}")
private String passWord;
@Value("${datasource.dynamic.driver-class-name}")
private String driverClassName;
@Value("${hospital.db.isolation}")
private boolean isolation;
@Override
public boolean preHandle(HttpServletRequest request, HttpServletResponse response, Object handler)
throws IOException {
if (isolation){
String hospitalId = request.getHeader("hospitalId");
if(StringUtils.isNotBlank(hospitalId)){
String dataType="wit120_"+request.getHeader("hospitalId");
DynamicDataSource dynamicDataSource = DynamicDataSource.getInstance();
Map<Object,Object> dataSourceMap = dynamicDataSource.getDataSourceMap();
if (dataSourceMap.get(dataType)==null){
DruidDataSource datasource = new DruidDataSource();
String dbUrl = this.url.replace("wit120",dataType);
datasource.setUrl(dbUrl);
datasource.setUsername(this.userName);
datasource.setPassword(this.passWord);
datasource.setDriverClassName(this.driverClassName);
datasource.setInitialSize(5);
datasource.setMinIdle(5);
datasource.setMaxActive(20);
datasource.setMaxWait(60000);
datasource.setTimeBetweenEvictionRunsMillis(60000);
datasource.setMinEvictableIdleTimeMillis(300000);
datasource.setValidationQuery("SELECT 1 FROM DUAL ");
datasource.setTestWhileIdle(true);
datasource.setTestOnBorrow(false);
datasource.setTestOnReturn(false);
datasource.setPoolPreparedStatements(true);
datasource.setMaxPoolPreparedStatementPerConnectionSize(20);
try {
datasource.setFilters("config,stat,wall,log4j");
datasource.setConnectionProperties("allowMultiQueries=true;druid.stat.mergeSql=true;druid.stat.slowSqlMillis=3000;config.decrypt="+useEncryptedPassword);
dynamicDataSource.setTargetDataSources(dataSourceMap);
datasource.getConnection();
//加载数据源
dataSourceMap.put(dataType,datasource);
} catch (Exception e) {
datasource.close();
refuse(response);
return false;
}
}
//存储医院标记
HospitalContextHolder.setHospitalId(hospitalId);
}
}
return true;
}
/**
* 拒绝的响应处理
*
* @param response
* @throws IOException
*/
private void refuse(HttpServletResponse response) throws IOException {
response.setContentType("application/json;charset=UTF-8");
response.getOutputStream().write(ResultUtil.resultFailed(BaseBizError.HOSPTAIL_Id_NOT_EXIST).toJsonString().getBytes("UTF-8"));
}
}
基本上 就实现了，动态数据源的加载，切换
网友评论