用户工具

站点工具


02-工程实践:大数据:opentsdb-hbase

CDH安装

opentsdb使用的hbase用cdh安装

要点:

  • 配置内网yum源及parcels仓库
  • master节点上安装cloudera-manager-server-db-2
  • 先启动cloudera-manager-server-db-2, 设置JAVAHOME,启动cloudera-maanger-server * 通过浏览器访问http://master_ip:7180 * 使用内网parcels及yum源

    LZO支持

    准备内网gplextras5 parcels仓库,cdh安装此parcels,并在io.compression.codecs属性值中追加如下值: <code> com.hadoop.compression.lzo.LzoCodec com.hadoop.compression.lzo.LzopCodec </code>

    create.sh脚本

    设置HBASEHOME

export HBASE_HOME=/letv/cloudera/parcels/CDH-5.7.6-1.cdh5.7.6.p0.6/lib/hbase/

添加TTL

create.sh
#!/bin/sh
# Small script to setup the HBase tables used by OpenTSDB.
 
test -n "$HBASE_HOME" || {
  echo >&2 'The environment variable HBASE_HOME must be set'
  exit 1
}
test -d "$HBASE_HOME" || {
  echo >&2 "No such directory: HBASE_HOME=$HBASE_HOME"
  exit 1
}
 
TSDB_TABLE=${TSDB_TABLE-'tsdb'}
UID_TABLE=${UID_TABLE-'tsdb-uid'}
TREE_TABLE=${TREE_TABLE-'tsdb-tree'}
META_TABLE=${META_TABLE-'tsdb-meta'}
BLOOMFILTER=${BLOOMFILTER-'ROW'}
# LZO requires lzo2 64bit to be installed + the hadoop-gpl-compression jar.
COMPRESSION=${COMPRESSION-'LZO'}
# All compression codec names are upper case (NONE, LZO, SNAPPY, etc).
COMPRESSION=`echo "$COMPRESSION" | tr a-z A-Z`
 
case $COMPRESSION in
  (NONE|LZO|GZIP|SNAPPY)  :;;  # Known good.
  (*)
    echo >&2 "warning: compression codec '$COMPRESSION' might not be supported."
    ;;
esac
 
# HBase scripts also use a variable named `HBASE_HOME', and having this
# variable in the environment with a value somewhat different from what
# they expect can confuse them in some cases.  So rename the variable.
hbh=$HBASE_HOME
unset HBASE_HOME
exec "$hbh/bin/hbase" shell <<EOF
create '$UID_TABLE',
  {NAME => 'id', COMPRESSION => '$COMPRESSION', BLOOMFILTER => '$BLOOMFILTER'},
  {NAME => 'name', COMPRESSION => '$COMPRESSION', BLOOMFILTER => '$BLOOMFILTER'}
 
create '$TSDB_TABLE',
  {NAME => 't', VERSIONS => 1, COMPRESSION => '$COMPRESSION', BLOOMFILTER => '$BLOOMFILTER', TTL => '604800'}
 
create '$TREE_TABLE',
  {NAME => 't', VERSIONS => 1, COMPRESSION => '$COMPRESSION', BLOOMFILTER => '$BLOOMFILTER'}
 
create '$META_TABLE',
  {NAME => 'name', COMPRESSION => '$COMPRESSION', BLOOMFILTER => '$BLOOMFILTER'}
EOF
02-工程实践/大数据/opentsdb-hbase.txt · 最后更改: 2020/04/07 06:34 由 annhe