《MongoDB极简教程》第一章 NoSQL简史 & M

作者: 光剑书架上的书 | 来源:发表于2017-03-19 02:59 被阅读389次

    MongoDB 是一款开源的文档数据库,并且是业内领先的 NoSQL 数据库,用 C++ 编写而成。

    NoSQL

    (NoSQL = Not Only SQL ),意即"不仅仅是SQL"。

    在现代的计算系统上每天网络上都会产生庞大的数据量。
    这些数据有很大一部分是由关系数据库管理系统(RDMBSs)来处理。 1970年 E.F.Codd's提出的关系模型的论文 "A relational model of data for large shared data banks",这使得数据建模和应用程序编程更加简单。

    通过应用实践证明,关系模型是非常适合于客户服务器编程,远远超出预期的利益,今天它是结构化数据存储在网络和商务应用的主导技术。

    NoSQL 是一项全新的数据库革命性运动,早期就有人提出,发展至2009年趋势越发高涨。NoSQL的拥护者们提倡运用非关系型的数据存储,相对于铺天盖地的关系型数据库运用,这一概念无疑是一种全新的思维的注入。

    RDBMS vs NoSQL
    RDBMS

    • 高度组织化结构化数据
    • 结构化查询语言(SQL) (SQL)
    • 数据和关系都存储在单独的表中。
    • 数据操纵语言,数据定义语言
    • 严格的一致性
    • 基础事务
      NoSQL
    • 代表着不仅仅是SQL
    • 没有声明性查询语言
    • 没有预定义的模式
      -键 - 值对存储,列存储,文档存储,图形数据库
    • 最终一致性,而非ACID属性
    • 非结构化和不可预知的数据
    • CAP定理
    • 高性能,高可用性和可伸缩性

    NoSQL 简史

    NoSQL一词最早出现于1998年,是Carlo Strozzi开发的一个轻量、开源、不提供SQL功能的关系数据库。

    2009年,Last.fm的Johan Oskarsson发起了一次关于分布式开源数据库的讨论[2],来自Rackspace的Eric Evans再次提出了NoSQL的概念,这时的NoSQL主要指非关系型、分布式、不提供ACID的数据库设计模式。

    2009年在亚特兰大举行的"no:sql(east)"讨论会是一个里程碑,其口号是"select fun, profit from real_world where relational=false;"。因此,对NoSQL最普遍的解释是"非关联型的",强调Key-Value Stores和文档数据库的优点,而不是单纯的反对RDBMS。

    CAP定理(CAP theorem)

    在计算机科学中, CAP定理(CAP theorem), 又被称作 布鲁尔定理(Brewer's theorem), 它指出对于一个分布式计算系统来说,不可能同时满足以下三点:

    一致性(Consistency) (所有节点在同一时间具有相同的数据)
    可用性(Availability) (保证每个请求不管成功或者失败都有响应)
    分隔容忍(Partition tolerance) (系统中任意信息的丢失或失败不会影响系统的继续运作)

    CAP理论的核心是:一个分布式系统不可能同时很好的满足一致性,可用性和分区容错性这三个需求,最多只能同时较好的满足两个。
    因此,根据 CAP 原理将 NoSQL 数据库分成了满足 CA 原则、满足 CP 原则和满足 AP 原则三 大类:

    CA - 单点集群,满足一致性,可用性的系统,通常在可扩展性上不太强大。
    CP - 满足一致性,分区容忍性的系统,通常性能不是特别高。
    AP - 满足可用性,分区容忍性的系统,通常可能对一致性要求低一些。
    
    cap-theoram-image

    NoSQL的优点/缺点

    优点:

    • 高可扩展性
    • 分布式计算
    • 低成本
    • 架构的灵活性,半结构化数据
    • 没有复杂的关系

    缺点:

    • 没有标准化
    • 有限的查询功能(到目前为止)
    • 最终一致是不直观的程序

    BASE

    BASE:Basically Available, Soft-state, Eventually Consistent。 由 Eric Brewer 定义。

    CAP理论的核心是:一个分布式系统不可能同时很好的满足一致性,可用性和分区容错性这三个需求,最多只能同时较好的满足两个。
    BASE是NoSQL数据库通常对可用性及一致性的弱要求原则:

    Basically Availble --基本可用
    Soft-state --软状态/柔性事务。 "Soft state" 可以理解为"无连接"的, 而 "Hard state" 是"面向连接"的
    Eventual Consistency --最终一致性 最终一致性, 也是是 ACID 的最终目的。

    MongoDB 特性&优势

    MongoDB 特性 优势
    事务支持 MongoDB 目前只支持单文档事务,需要复杂事务支持的场景暂时不适合
    灵活的文档模型 JSON 格式存储最接近真实对象模型,对开发者友好,方便快速开发迭代
    高可用复制集 满足数据高可靠、服务高可用的需求,运维简单,故障自动切换
    可扩展分片集群 海量数据存储,服务能力水平扩展
    高性能 mmapv1、wiredtiger、mongorocks(rocksdb)、in-memory 等多引擎支持满足各种场景需求
    强大的索引支持 地理位置索引可用于构建 各种 O2O 应用、文本索引解决搜索的需求、TTL索引解决历史数据自动过期的需求
    Gridfs 解决文件存储的需求
    aggregation & mapreduce 解决数据分析场景需求,用户可以自己写查询语句或脚本,将请求都分发到 MongoDB 上完成

    文档参考

    中文社区

    MongoDB 极简实践入门

    安装&环境配置

    下载:https://www.mongodb.com/download-center#community

    安装:
    https://docs.mongodb.com/manual/administration/install-community/

    解压,配置环境 变量PATH即可。

    MONGODB_HOME=/Users/jack/soft/mongodb-osx-x86_64-3.4.2
    export PATH=$PATH:$MONGODB_HOME/bin
    
    

    启动脚本

    cat start_mongodb.sh 
    #!/bin/bash
    source ~/.bashrc
    
    mongod --dbpath ~/dbdata
    

    启动服务进程

    ./start_mongodb.sh 
    
    2017-03-19T02:50:44.491+0800 I CONTROL  [initandlisten] MongoDB starting : pid=67062 port=27017 dbpath=/Users/jack/dbdata 64-bit host=ChenJackdeMacBook-Pro.local
    2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] db version v3.4.2
    2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] git version: 3f76e40c105fc223b3e5aac3e20dcd026b83b38b
    2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] allocator: system
    2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] modules: none
    2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] build environment:
    2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten]     distarch: x86_64
    2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten]     target_arch: x86_64
    2017-03-19T02:50:44.492+0800 I CONTROL  [initandlisten] options: { storage: { dbPath: "/Users/jack/dbdata" } }
    2017-03-19T02:50:44.494+0800 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=3584M,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
    2017-03-19T02:50:45.352+0800 I CONTROL  [initandlisten] 
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
    2017-03-19T02:50:45.748+0800 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/Users/jack/dbdata/diagnostic.data'
    2017-03-19T02:50:46.073+0800 I INDEX    [initandlisten] build index on: admin.system.version properties: { v: 2, key: { version: 1 }, name: "incompatible_with_version_32", ns: "admin.system.version" }
    2017-03-19T02:50:46.073+0800 I INDEX    [initandlisten]      building index using bulk method; build may temporarily use up to 500 megabytes of RAM
    2017-03-19T02:50:46.123+0800 I INDEX    [initandlisten] build index done.  scanned 0 total records. 0 secs
    2017-03-19T02:50:46.124+0800 I COMMAND  [initandlisten] setting featureCompatibilityVersion to 3.4
    2017-03-19T02:50:46.124+0800 I NETWORK  [thread1] waiting for connections on port 27017
    
    
    
    2017-03-19T02:51:24.533+0800 I NETWORK  [thread1] connection accepted from 127.0.0.1:49370 #1 (1 connection now open)
    2017-03-19T02:51:24.534+0800 I NETWORK  [conn1] received client metadata from 127.0.0.1:49370 conn1: { application: { name: "MongoDB Shell" }, driver: { name: "MongoDB Internal Client", version: "3.4.2" }, os: { type: "Darwin", name: "Mac OS X", architecture: "x86_64", version: "16.4.0" } }
    
    
    

    Terminal连接客户端

    $ mongo
    MongoDB shell version v3.4.2
    connecting to: mongodb://127.0.0.1:27017
    MongoDB server version: 3.4.2
    Server has startup warnings: 
    2017-03-19T02:50:45.352+0800 I CONTROL  [initandlisten] 
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: Access control is not enabled for the database.
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] **          Read and write access to data and configuration is unrestricted.
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] 
    2017-03-19T02:50:45.353+0800 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. Number of files is 256, should be at least 1000
    > 
    > 
    > show dbs;
    admin  0.000GB
    local  0.000GB
    
    

    强大的 MongoDB 管理工具

    MongoDB Shell Enhancements

    https://github.com/Jason-Chen-2017/mongo-hacker

    Installation

    npm install -g mongo-hacker
    

    Enhancements

    Basic UX

    • Sort document keys by default
    • Highlight querytime if verboseShell is enabled
      • In green if querytime is at or below slowms
      • In red if query time is above slowms
    • Default indent is 2 spaces instead of tab
      • Customizable by setting indent key of config
    • Verbose shell is enabled by default -- to disable: setVerboseShell(false)
    • Disable notfication of "Type 'it' for more"
    • Custom prompt: hostname(process-version)[rs_status:set_name] db>
    • Always pretty print. You can still use default format by appending .ugly() to the end of db statement.
    • Show DBs has aligned columns, is sorted by database name and shows less significant digits (in master for Mongo 2.5/2.6)
    • Nicer sh.status() output (remove lastmod, take up less space, colorize chunk's shard)
    • Colorized query output for console/terminal windows supporting ANSI color codes.


      Colorized Output

    Additional shell commands

    The MongoDB shell offers various "shell commands" (sometimes referred to as "shell helpers" as well) that make interactive use of the shell much more convenient than proper, Javascript-only scripted use of the shell.

    To make interactive use of the MongoDB shell even more convenient, mongo-hacker adds the following shell commands:

    • count collections/count tables: count the number of collections in each of the mongo server's databases - by @pvdb
    • count documents/count docs: count the number of documents in all (non-system) collections in the database - by @pvdb
    • count indexes: list all collections and display the size of all indexes - by @cog-g

    Some of these commands have hidden features that can be enabled in the mongo-hacker config, to make the command output even more useful:

    • by changing the count_deltas setting to true in config.js, the count documents command will also print out the change in the number of documents since the last count - by @pvdb

    API Additions

    Scripting

    Get a list of database names: (by @pvdb)

    db.getMongo().getDatabaseNames()
    

    (note that this method is similar - functionality-wise and usage-wise - to the existing db.getCollectionNames() API method and allows for advanced, cross-database scripting in the MongoDB shell)

    General

    Filter for a collection of documents:

    db.collection.filter(<criteria>)
    

    One for finding a single document:

    db.collection.find({ ... }).one() == db.collection.findOne({ ... })
    

    Select for selecting fields to return (projection):

    db.collection.find({ ... }).select({ name: 1 })
    

    Reverse for descending sort by insertion order (default) or arbitrary field:

    db.collection.find({ ... }).reverse()
    db.collection.find({ ... }).reverse('createDate')
    

    Last for finding last inserted document (default) or document last by given field:

    db.collection.find({ ... }).last()
    db.collection.find({ ... }).last('createDate')
    

    Update, Replace, Upsert and Remove can be called on a DBQuery Object:

    db.collection.find({ ... }).update({ ... })  // multi update
    db.collection.find({ ... }).replace({ ... }) // single replacement
    db.collection.find({ ... }).upsert({ ... })  // single upsert
    db.collection.find({ ... }).remove()         // multi remove
    

    Sort, limit, and skip through multi updates and removes:

    db.collection.find({ ... }).limit(7).update({ ... })
    db.collection.find({ ... }).sort({ ... }).skip(1).limit(3).update({ ... })
    db.collection.find({ ... }).limit(3).remove()
    

    Note: The performance of multi updates involving a skip or limit may be worse than those without those specifications due to there being absolutely no native support for this feature in MongoDB itself. It should be understood by the user of this software that use of this feature (by calling update on a cursor rather than a collection) is advanced and experimental. The option to do this sort of operation is purely additive to the MongoDB experience with MongoHacker and usage of it is in no way required. Furthermore, its inclusion in this enhancement does not effect the operation of updates invoked through collections and, in practice, is insanely useful.

    Aggregation Framework

    The aggregation framework is now fluent as well. You can use it as currently documented or via the chainable methods.

    Calling aggregate without an array of operations or $operations will make it a match.

    // matches every document
    db.collection.aggregate()
    db.collection.aggregate({})
    
    // matches documents where the "a" is equal to 1
    db.collection.aggregate({a: 1})
    
    // matches documents where "a" is greater than 7
    db.collection.aggregate({a: {$gt: 7}})
    

    Additional methods can then be chained on top of the inital match in order to make more complicated aggregations.

    // Match and project
    db.collection.aggregate(<querydoc>).project(<projection>)
    db.collection.aggregate({a: 1}).project({a: 1, _id: 0})
    
    // Match, group and sort
    db.collection.aggregate({<match>}).group({<group>}).sort({<sort>})
    db.test.aggregate().group({_id: '$a', 'sum': {'$sum': 1}}).sort({sum: -1})
    

    Data Generation

    For easy and simple random data generation you can utilise these methods below. You can use any of these functions in a loop. For example:

    // Inserts 20 documents with random data. 
    for (i=1; i<21; i++) { 
        db.collection.insert(
                {
                 word: randomWord(), 
                 number: randomNumber(), 
                 date: randomDate() 
                }
        ); 
    }
    
    randomWord

    You can specify the length of each word, the number of words, and an optional seeded word in a sentence randomly. Use the optional seed parameter for testing text search.

    randomWord(length=5, words=1, seed=undefined)

    // Inserts a random sentence consisting of 5 letters per word, 5 words in total, 
    // with a probability to insert the word 'needle' in the sentence
    db.collection.insert( { words: randomWord(5, 5, 'needle') } )
    
    // Inserts a random word consisting of 16 letters
    db.collection.insert( { words: randomWord(16) } )
    
    randomNumber

    You can specify maximum number to be randomly generated (exclusive)

    randomNumber(max=100)

    // Inserts a random number in the range of 0 or 1. 
    db.collection.insert( { number: randomNumber(2) } )
    
    // Inserts a random number in the range of 0 or 999. 
    db.collection.insert( { number: randomNumber(1000) } )
    
    
    randomDate

    You can specify start and end dates range to be randomly generated. (exclusive)

    randomDate(start= <2 years ago> , end=Date() )

    // Inserts a random date object in the range of 1st January 2016 to 1st February 2016
    db.collection.insert( { date: randomDate(ISODate("2016-01-01T00:00:00"), ISODate("2016-02-01T00:00:00")) })
    
    // If today is 19th May 2016 and you specify only the start of the day, 
    // this will generate random date object between 00:00:00 to current time.  
    db.collection.insert( { date: randomDate(ISODate("2016-05-19T00:00:00")) })
    

    Helpers

    General Shell Helpers

    • findCommand('search') list commands that match the search string

    Aggregation Framework Helpers -- on collections

    • Group and Count: gcount(group_field, filter)
    • Group and Sum: gsum(group_field, sum_field, filter)
    • Group and Average: gavg(group_field, avg_field, filter)

    Run function on some/all databases

    runOnDbs(/db_names_regexp/, function(db) {
        // callback is ran for each database which name matches regular expression
        // db is that selected database
    });
    

    mongo-express

    https://github.com/Jason-Chen-2017/mongo-express

    Web-based MongoDB admin interface, written with Node.js and express

    终端查看的效果:

    jack@jacks-MacBook-Air:~$ mongo
    MongoDB shell version: 3.2.4
    connecting to: test
    Mongo-Hacker 0.0.14
    Server has startup warnings: 
    2017-03-13T10:31:13.023+0800 I CONTROL  [initandlisten] ** WARNING: You are running this process as the root user, which is not recommended.
    2017-03-13T10:31:13.023+0800 I CONTROL  [initandlisten] 
    jacks-MacBook-Air(mongod-3.2.4) test> 
    jacks-MacBook-Air(mongod-3.2.4) test> 
    jacks-MacBook-Air(mongod-3.2.4) test> db
    test
    jacks-MacBook-Air(mongod-3.2.4) test> show dbs;
    admin      → 0.000GB
    local      → 0.000GB
    restfiddle → 0.001GB
    jacks-MacBook-Air(mongod-3.2.4) test> use restfiddle;
    switched to db restfiddle
    jacks-MacBook-Air(mongod-3.2.4) restfiddle> show collections;
    EntityAuth        → 0.000MB / 0.004MB
    activityLog       → 0.001MB / 0.035MB
    baseNode          → 0.021MB / 0.043MB
    config            → 0.000MB / 0.016MB
    conversation      → 0.017MB / 0.035MB
    httpRequestHeader → 0.005MB / 0.016MB
    oAuth2            → 0.002MB / 0.016MB
    project           → 0.002MB / 0.031MB
    rfRequest         → 0.017MB / 0.035MB
    rfResponse        → 0.036MB / 0.047MB
    runnerLog         → 0.000MB / 0.016MB
    tag               → 0.000MB / 0.016MB
    user              → 0.000MB / 0.035MB
    workspace         → 0.001MB / 0.031MB
    jacks-MacBook-Air(mongod-3.2.4) restfiddle> l
    2017-03-21T14:18:55.202+0800 E QUERY    [thread1] ReferenceError: l is not defined :
    @(shell):1:1
    
    jacks-MacBook-Air(mongod-3.2.4) restfiddle> db.runnerLog.find()
    {
      "_id": ObjectId("575fad7dba637304fffbae85"),
      "_class": "com.restfiddle.entity.RunnerLog",
      "status": "ACTIVE",
      "version": NumberLong("0")
    }
    {
      "_id": ObjectId("575fad92ba637304fffbae86"),
      "_class": "com.restfiddle.entity.RunnerLog",
      "status": "ACTIVE",
      "version": NumberLong("0")
    }
    {
      "_id": ObjectId("575fad9cba637304fffbae87"),
      "_class": "com.restfiddle.entity.RunnerLog",
      "status": "ACTIVE",
      "version": NumberLong("0")
    }
    Fetched 3 record(s) in 7ms
    jacks-MacBook-Air(mongod-3.2.4) restfiddle> 
    
    

    mongo-express的效果:

    螢幕快照 2017-03-21 14.25.03.png 螢幕快照 2017-03-21 14.25.57.png

    相关文章

      网友评论

        本文标题:《MongoDB极简教程》第一章 NoSQL简史 & M

        本文链接:https://www.haomeiwen.com/subject/gdfmnttx.html