sabato 17 settembre 2016

Approccio a MongoDB su CentOS

Seguono alcuni appunti a riguardo dell'installazione e dell'utilizzo di MongoDB su CentOS 7.

Gli appunti sono basati sulla documentazione che si trova all'indirizzo http://docs.mongodb.com.

Installazione

[root@mongodbhost1 ~]# ed
f /etc/yum.repos.d/mongodb-org-3.2.repo
/etc/yum.repos.d/mongodb-org-3.2.repo
a
[mongodb-org-3.2]
name=MongoDB Repository
baseurl=https://repo.mongodb.org/yum/redhat/$releasever/mongodb-org/3.2/x86_64/
gpgcheck=1
enabled=1
gpgkey=https://www.mongodb.org/static/pgp/server-3.2.asc
.
w
202
q
[root@mongodbhost1 ~]# yum search mongodb
...
mongodb-org.x86_64 : MongoDB open source document-oriented database system (metapackage)
mongodb-org-mongos.x86_64 : MongoDB sharded cluster query router
mongodb-org-server.x86_64 : MongoDB database server
mongodb-org-shell.x86_64 : MongoDB shell client
mongodb-org-tools.x86_64 : MongoDB tools

  Name and summary matches only, use "search all" for everything.
[root@mongodbhost1 ~]# yum install -y mongodb-org
...
[root@mongodbhost1 ~]#

Avvio

[root@mongodbhost1 ~]# service mongod start
Starting mongod (via systemctl):                           [  OK  ]
[root@mongodbhost1 ~]# ps -efa | grep mongo
mongod   17481     1  8 14:47 ?        00:00:00 /usr/bin/mongod -f /etc/mongod.conf
root     17500 10332  0 14:47 pts/0    00:00:00 grep --color=auto mongo
[root@mongodbhost1 ~]# grep STARTED /var/log/mongodb/mongod.log|tail -1
2016-08-17T14:47:21.405+0200 I CONTROL  [main] ***** SERVER RESTARTED *****
[root@mongodbhost1 ~]# 

Spegnimento

[root@mongodbhost1 ~]# service mongod stop
Stopping mongod (via systemctl):                           [  OK  ]
[root@mongodbhost1 ~]# ps -efa | grep mongo
root     17561 10332  0 14:58 pts/0    00:00:00 grep --color=auto mongo
[root@mongodbhost1 ~]# grep dbexit /var/log/mongodb/mongod.log|tail -1
2016-08-17T14:58:12.586+0200 I CONTROL  [signalProcessingThread] dbexit:  rc: 0
[root@mongodbhost1 ~]# 

Avvio automatico

[root@mongodbhost1 ~]# chkconfig mongod on
[root@mongodbhost1 ~]# chkconfig

Note: This output shows SysV services only and does not include native
      systemd services. SysV configuration data might be overridden by native
      systemd configuration.

      If you want to list systemd services use 'systemctl list-unit-files'.
      To see services enabled on particular target use
      'systemctl list-dependencies [target]'.

mongod          0:off 1:off 2:on 3:on 4:on 5:on 6:off
netconsole      0:off 1:off 2:off 3:off 4:off 5:off 6:off
network         0:off 1:off 2:on 3:on 4:on 5:on 6:off
[root@mongodbhost1 ~]# 

Ottimizzazioni del kernel Linux

[root@mongodbhost1 ~]# cat /var/log/mongodb/mongod.log
...
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] 
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/enabled is 'always'.
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] 
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] ** WARNING: /sys/kernel/mm/transparent_hugepage/defrag is 'always'.
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] **        We suggest setting it to 'never'
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] 
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 4096 processes, 64000 files. Number of processes should be at least 32000 : 0.5 times number of files.
2016-08-17T14:47:22.003+0200 I CONTROL  [initandlisten] 
...
[root@mongodbhost1 ~]# cat /sys/kernel/mm/transparent_hugepage/enabled
[always] madvise never
[root@mongodbhost1 ~]# cat /sys/kernel/mm/transparent_hugepage/defrag
[always] madvise never
[root@mongodbhost1 ~]# 

### create this configuration. as of
###    https://docs.mongodb.com/manual/tutorial/transparent-huge-pages/
### and
###    https://serverfault.com/questions/688392/disable-thp-and-thp-defrag-on-centos-7-ec2-instance
###

[root@mongodbhost1 no-thp]# pwd
/etc/tuned/no-thp
[root@mongodbhost1 no-thp]# ls -l
total 8
-rwxr-xr-x. 1 root root 161 Aug 17 15:24 disable_transparent_hugepages_defrag.sh
-rw-r--r--. 1 root root 120 Aug 17 15:23 tuned.conf
[root@mongodbhost1 no-thp]# cat tuned.conf
[main]
include=virtual-guest

[vm]
transparent_hugepages=never

[script]
script=disable_transparent_hugepages_defrag.sh
[root@mongodbhost1 no-thp]# cat disable_transparent_hugepages_defrag.sh
#!/bin/sh

. /usr/lib/tuned/functions

start() {
    echo never > /sys/kernel/mm/transparent_hugepage/defrag
    return 0
}

stop() {
    return 0
}

process $@
[root@mongodbhost1 no-thp]# tuned-adm profile no-thp
[root@mongodbhost1 no-thp]# cat /sys/kernel/mm/transparent_hugepage/enabled 
always madvise [never]
[root@mongodbhost1 no-thp]# cat /sys/kernel/mm/transparent_hugepage/defrag 
always madvise [never]
[root@mongodbhost1 no-thp]# 



Estensione del numero di processi concessi

### All'avvio, nei log, riceviamo uno warning relativo al limite sul numero dei processi concessi all'utente mongod, limite che è troppo restrittivo.
[root@mongodbhost1 ~]# cat /var/log/mongodb/mongod.log
...
2016-09-17T13:30:01.551+0200 I CONTROL  [main] ***** SERVER RESTARTED *****
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten] MongoDB starting : pid=2396 port=27017 dbpath=/var/lib/mongo 64-bit host=mongodbhost1.localdomain
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten] db version v3.2.9
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten] git version: 22ec9e93b40c85fc7cae7d56e7d6a02fd811088c
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten] allocator: tcmalloc
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten] modules: none
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten] build environment:
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten]     distmod: rhel70
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten]     distarch: x86_64
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten]     target_arch: x86_64
2016-09-17T13:30:01.602+0200 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" }, storage: { dbPath: "/var/lib/mongo", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2016-09-17T13:30:01.646+0200 I -        [initandlisten] Detected data files in /var/lib/mongo created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-09-17T13:30:01.646+0200 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-09-17T13:30:02.393+0200 I CONTROL  [initandlisten] 
2016-09-17T13:30:02.393+0200 I CONTROL  [initandlisten] ** WARNING: soft rlimits too low. rlimits set to 4096 processes, 64000 files. Number of processes should be at least 32000 : 0.5 times number of files.
2016-09-17T13:30:02.442+0200 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongo/diagnostic.data'
2016-09-17T13:30:02.443+0200 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
2016-09-17T13:30:02.444+0200 I NETWORK  [initandlisten] waiting for connections on port 27017
...
### Quindi estendiamo il limite menzionato precedentemente.
[root@mongodbhost1 ~]# ed /etc/security/limits.d/20-nproc.conf 
191
,p
# Default limit for number of user's processes to prevent
# accidental fork bombs.
# See rhbz #432903 for reasoning.

*          soft    nproc     4096
root       soft    nproc     unlimited
a
mongod     soft    nproc     64000
.
w
226
q
[root@mongodbhost1 ~]# 
### Ci siamo liberati dello warning.
2016-09-17T13:44:02.349+0200 I CONTROL  [main] ***** SERVER RESTARTED *****
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten] MongoDB starting : pid=2532 port=27017 dbpath=/var/lib/mongo 64-bit host=mongodbhost1.localdomain
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten] db version v3.2.9
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten] git version: 22ec9e93b40c85fc7cae7d56e7d6a02fd811088c
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten] OpenSSL version: OpenSSL 1.0.1e-fips 11 Feb 2013
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten] allocator: tcmalloc
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten] modules: none
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten] build environment:
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten]     distmod: rhel70
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten]     distarch: x86_64
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten]     target_arch: x86_64
2016-09-17T13:44:02.354+0200 I CONTROL  [initandlisten] options: { config: "/etc/mongod.conf", net: { bindIp: "127.0.0.1", port: 27017 }, processManagement: { fork: true, pidFilePath: "/var/run/mongodb/mongod.pid" }, storage: { dbPath: "/var/lib/mongo", journal: { enabled: true } }, systemLog: { destination: "file", logAppend: true, path: "/var/log/mongodb/mongod.log" } }
2016-09-17T13:44:02.369+0200 I -        [initandlisten] Detected data files in /var/lib/mongo created by the 'wiredTiger' storage engine, so setting the active storage engine to 'wiredTiger'.
2016-09-17T13:44:02.369+0200 I STORAGE  [initandlisten] wiredtiger_open config: create,cache_size=1G,session_max=20000,eviction=(threads_max=4),config_base=false,statistics=(fast),log=(enabled=true,archive=true,path=journal,compressor=snappy),file_manager=(close_idle_time=100000),checkpoint=(wait=60,log_size=2GB),statistics_log=(wait=0),
2016-09-17T13:44:02.816+0200 I FTDC     [initandlisten] Initializing full-time diagnostic data capture with directory '/var/lib/mongo/diagnostic.data'
2016-09-17T13:44:02.816+0200 I NETWORK  [initandlisten] waiting for connections on port 27017
2016-09-17T13:44:02.817+0200 I NETWORK  [HostnameCanonicalizationWorker] Starting hostname canonicalization worker
[root@mongodbhost1 ~]# 

File di configurazione e di log predefiniti

[root@mongodbhost1 ~]# ps -efa | grep mongod
mongod    2532     1  0 13:44 ?        00:00:00 /usr/bin/mongod -f /etc/mongod.conf
root      2557  2446  0 13:46 pts/0    00:00:00 grep --color=auto mongod
[root@mongodbhost1 ~]# cat  /etc/mongod.conf
# mongod.conf

# for documentation of all options, see:
#   http://docs.mongodb.org/manual/reference/configuration-options/

# where to write logging data.
systemLog:
  destination: file
  logAppend: true
  path: /var/log/mongodb/mongod.log

# Where and how to store data.
storage:
  dbPath: /var/lib/mongo
  journal:
    enabled: true
#  engine:
#  mmapv1:
#  wiredTiger:

# how the process runs
processManagement:
  fork: true  # fork and run in background
  pidFilePath: /var/run/mongodb/mongod.pid  # location of pidfile

# network interfaces
net:
  port: 27017
  bindIp: 127.0.0.1  # Listen to local interface only, comment to listen on all interfaces.


#security:

#operationProfiling:

#replication:

#sharding:

## Enterprise-Only Options

#auditLog:

#snmp:
[root@mongodbhost1 ~]# 

Continuando con il tutorial iniziale

Leggere https://docs.mongodb.com/getting-started/shell/import-data.