分享

在一台机器上部署swift存储和swauth认证(swift单独部署)

pig2 发表于 2015-2-21 10:45:41 [显示全部楼层] 回帖奖励 阅读模式 关闭右栏 0 27648

问题导读

1.如何创建swift用户?
2.存储节点设置需要注意什么问题?
3.swift_hash_path_suffix作用是什么?






1.  摘要说明
本文所涉及到的所有操作都是在Ubuntu Server 11.10 64位系统上通过验证。本文参考了Openstackswift的SAIO(Swift All In One)文档,具体信息如下:
Linux系统版本:Ubuntu Server11.10 64-bit oneiric
机器系统IP:   192.168.112.129
官方文档:     www.openstack.org
Swift版本:       1.4.8
Swauth版本:  1.0.4
2.  安装依赖包
sudo  apt-get --option Dpkg::Options::=--force-confold --assume-yes update
sudo  apt-get install pep8 pylint python-pip screen unzip wget psmisc git-core lsofopenssh-server vim-nox locate python-virtualenv python-unittest2 iputils-pingwget curl tcpdump euca2ools python-setuptools python-dev python-lxmlpython-pastescript python-pastedeploy python-paste sqlite3 python-pysqlite2python-sqlalchemy python-mysqldb python-webob python-greenlet python-routeslibldap2-dev libsasl2-dev python-bcrypt curl gcc python-configobjpython-coverage python-dev python-eventlet python-greenlet python-netifacespython-nose python-pastedeploy python-setuptools python-simplejson python-webobpython-xattr sqlite3 xfsprogs
3.  创建swift用户


  1. sudo useradd -mk /home/swift/ -s /bin/bash swift  
  2. sudo passwd swift    #为swift用户添加密码,在此我将其设为了swift  
  3. 编辑/etc/sudoer文件,在文件末尾添加如下代码  
  4. swift  ALL=(ALL) NOPASSWD:ALL  
复制代码
4.  下载swift和swauth源码 (在swift用户下操作)
  1. su swift      #切换到swift用户
  2. sudo mkdir /home/swift/openstack       #创建一个目录来存放
  3. cd /home/swift/openstack
  4. git  clone https://github.com/openstack/swift.git #下载swift
  5. cd  swift
  6. git checkout 1.4.8       #使用1.4.8版本,在swift目录下你可以使用git tag命令查看有多少个版本
  7. git clone https://github.com/gholt/swauth.git    #下载swauth
  8. cd swauth
  9. git checkout 1.0.4
复制代码


5.  安装swift和swauth  (在swift用户下操作)
  1. cd /home/swift/openstack/swift
  2. sudo python setup.py install --record file.txt
  3. #参数--record file.txt的作用是将所有文件被安装到的路径记录到#file.txt中,为了以后想卸载该软件的时候用,要想卸载该软件时,到#file.txt所在目录输入如下命令即可:sudo cat file.txt | xargs rm –rf
  4. cd /home/swift/openstack/swauth
  5. sudo python setup.py install --record file.txt
复制代码
6.  安装相关软件
  1. sudo apt-get install memcached
复制代码
7.  存储点的设置  (在swift用户下操作)
在这里有两种方式来设置存储点
1.  你的系统里有一个单独分区,使用此分区来做存储点,在这里假设系统中有/dev/sdb1(注:这里根据你自己系统的情况而定)这个分区未被使用,我们用它来做存储点。


  1. sudo mkfs.xfs -i size=1024 /dev/sdb1   #以xfs方式格式化分区
  2. sudo chmod a+w /etc/fstab
  3. sudo echo “/dev/sdb1 /mnt/sdb1 xfs noatime,nodiratime nobarrier,logbufs=8 0 0”>> /etc/fstab        #系统启动时自动挂载
  4. sudo mkdir /mnt/sdb1                #在/mnt下创建sdb1挂载目录
  5. sudo mount /mnt/sdb1                #挂载
  6. sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
  7. sudo chown swift:swift /mnt/sdb1/*
  8. sudo mkdir /srv
  9. for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done #创建连接
  10. sudo mkdir -p /etc/swift/object-server /etc/swift/container-server/etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3/srv/4/node/sdb4 /var/run/swift        #创建相关目录
  11. sudo chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift
  12. 在/etc/rc.local的exit 0之前加入下列三行
  13. mkdir /var/run/swift
  14. chown swift:swift /var/run/swift
  15. chmod a+w /var/run/swift
复制代码
2.  如果系统中没有单独未使用的分区则可以创建一个回环设备来做存储点(在此我使用的就是此方法)
  1. sudo mkdir /srv
  2. sudo dd if=/dev/zero of=/srv/swift-disk bs=1024 count=0 seek=1000000   #这个命令是在/srv/下创建一个名为swift-disk的存
  3. #储区,你可以改变seek的大小来改变swift-disk的大小
  4. sudo mkfs.xfs -i size=1024 /srv/swift-disk
  5. sudo chmod a+w /etc/fstab
  6. sudo echo “/srv/swift-disk /mnt/sdb1 xfs loop,noatime,nodiratime,nobarrier,logbufs=80 0” >> /etc/fstab #系统启动
  7. #时自动挂载
  8. sudo mkdir /mnt/sdb1                #在/mnt下创建sdb1挂载目录
  9. sudo mount /mnt/sdb1                #挂载
  10. sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
  11. sudo chown swift:swift /mnt/sdb1/*
  12. for x in {1..4}; do sudo ln -s /mnt/sdb1/$x /srv/$x; done #创建连接
  13. sudo mkdir -p /etc/swift/object-server /etc/swift/container-server/etc/swift/account-server /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3/srv/4/node/sdb4 /var/run/swift        #创建相关目录
  14. sudo chown -R swift:swift /etc/swift /srv/[1-4]/ /var/run/swift
  15. 在/etc/rc.local的exit 0之前加入下列三行
  16. mkdir /var/run/swift
  17. chown swift:swift /var/run/swift
  18. chmod a+w /var/run/swift
复制代码
8.  设置rsync
  1. sudo touch /etc/rsyncd.conf    #在/etc/下创建rsyncd.conf
  2. sudo chmod a+w /etc/rsyncd.conf
  3. 将以下配置信息写入/etc/rsyncd.conf中
  4. uid =swift
  5. gid =swift
  6. logfile = /var/log/rsyncd.log
  7. pidfile = /var/run/rsyncd.pid
  8. address= 127.0.0.1
  9. [account6012]
  10. maxconnections = 25
  11. path =/srv/1/node/
  12. readonly = false
  13. lockfile = /var/lock/account6012.lock
  14. [account6022]
  15. maxconnections = 25
  16. path =/srv/2/node/
  17. readonly = false
  18. lockfile = /var/lock/account6022.lock
  19. [account6032]
  20. maxconnections = 25
  21. path =/srv/3/node/
  22. readonly = false
  23. lockfile = /var/lock/account6032.lock
  24. [account6042]
  25. maxconnections = 25
  26. path =/srv/4/node/
  27. readonly = false
  28. lockfile = /var/lock/account6042.lock
  29. [container6011]
  30. maxconnections = 25
  31. path =/srv/1/node/
  32. readonly = false
  33. lockfile = /var/lock/container6011.lock
  34. [container6021]
  35. maxconnections = 25
  36. path =/srv/2/node/
  37. readonly = false
  38. lockfile = /var/lock/container6021.lock
  39. [container6031]
  40. maxconnections = 25
  41. path =/srv/3/node/
  42. readonly = false
  43. lockfile = /var/lock/container6031.lock
  44. [container6041]
  45. maxconnections = 25
  46. path =/srv/4/node/
  47. readonly = false
  48. lockfile = /var/lock/container6041.lock
  49. [object6010]
  50. maxconnections = 25
  51. path =/srv/1/node/
  52. readonly = false
  53. lockfile = /var/lock/object6010.lock
  54. [object6020]
  55. maxconnections = 25
  56. path =/srv/2/node/
  57. readonly = false
  58. lockfile = /var/lock/object6020.lock
  59. [object6030]
  60. maxconnections = 25
  61. path =/srv/3/node/
  62. readonly = false
  63. lockfile = /var/lock/object6030.lock
  64. [object6040]
  65. maxconnections = 25
  66. path =/srv/4/node/
  67. readonly = false
  68. lockfile = /var/lock/object6040.lock
  69. 编辑/etc/default/rsync:
  70. 将RSYNC_ENABLE设置为true
  71. 重启服务:
  72. sudo servicersync restart
复制代码


9.  设置日志文件(可选)

  1. sudo touch /etc/rsyslog.d/10-swift.conf
  2. sudo chmod a+w /etc/rsyslog.d/10-swift.conf
复制代码

  1. 在该文件中添加如下配置信息
  2. #Uncomment the following to have a log containing all logs together
  3. #local1,local2,local3,local4,local5.*   /var/log/swift/all.log
  4. #Uncomment the following to have hourly proxy logs for stats processing
  5. #$templateHourlyProxyLog,"/var/log/swift/hourly/%$YEAR%%$MONTH%%$DAY%%$HOUR%"
  6. #local1.*;local1.!notice?HourlyProxyLog
  7. local1.*;local1.!notice/var/log/swift/proxy.log
  8. local1.notice           /var/log/swift/proxy.error
  9. local1.*                ~
  10. local2.*;local2.!notice/var/log/swift/storage1.log
  11. local2.notice           /var/log/swift/storage1.error
  12. local2.*                ~
  13. local3.*;local3.!notice/var/log/swift/storage2.log
  14. local3.notice           /var/log/swift/storage2.error
  15. local3.*                ~
  16. local4.*;local4.!notice/var/log/swift/storage3.log
  17. local4.notice           /var/log/swift/storage3.error
  18. local4.*                ~
  19. local5.*;local5.!notice/var/log/swift/storage4.log
  20. local5.notice           /var/log/swift/storage4.error
  21. local5.*   
复制代码

  1. 修改/etc/rsyslog.conf:
  2. 将$PrivDropToGroup  syslog改为$PrivDropToGroup adm
  3. 执行如下命令
  4. sudo mkdir -p /var/log/swift/hourly
  5. sudo chown -R syslog.adm /var/log/swift
  6. sudo service rsyslog restart
复制代码
10. 配置各个节点(以下文件不存在的话需要自己创建)
sudo chmod a+w -R /etc/swift/*

  1. 1.  /etc/swift/proxy-server.conf
  2. [DEFAULT]
  3. bind_port= 8080
  4. user =swift
  5. log_facility= LOG_LOCAL1
  6. [pipeline:main]
  7. pipeline= healthcheck cache swauth proxy-server
  8. [app:proxy-server]
  9. use =egg:swift#proxy
  10. allow_account_management= true
  11. account_autocreate= true
  12. [filter:swauth]
  13. use =egg:swauth#swauth
  14. #Highly recommended to change this.
  15. super_admin_key= swauthkey
  16. default_swift_cluster= local#http://127.0.0.1:8080/auth/v1.0
  17. [filter:healthcheck]
  18. use =egg:swift#healthcheck
  19. [filter:cache]
  20. use =egg:swift#memcache
  21. 2.  /etc/swift/swift.conf
  22. [swift-hash]
  23. #random unique string that can never change (DO NOT LOSE)
  24. swift_hash_path_suffix= cynric   #cynric是可以随意更改的字符串
  25. 3.  /etc/swift/account-server/1.conf
  26. [DEFAULT]
  27. devices= /srv/1/node
  28. mount_check= false
  29. bind_port= 6012
  30. user =swift
  31. log_facility= LOG_LOCAL2
  32. [pipeline:main]
  33. pipeline= account-server
  34. [app:account-server]
  35. use =egg:swift#account
  36. [account-replicator]
  37. vm_test_mode= yes
  38. [account-auditor]
  39. [account-reaper]
  40. 4.  /etc/swift/account-server/2.conf
  41. [DEFAULT]
  42. devices= /srv/2/node
  43. mount_check= false
  44. bind_port= 6022
  45. user =swift
  46. log_facility= LOG_LOCAL3
  47. [pipeline:main]
  48. pipeline= account-server
  49. [app:account-server]
  50. use =egg:swift#account
  51. [account-replicator]
  52. vm_test_mode= yes
  53. [account-auditor]
  54. [account-reaper]
  55. 5.  /etc/swift/account-server/3.conf
  56. [DEFAULT]
  57. devices= /srv/3/node
  58. mount_check= false
  59. bind_port= 6032
  60. user =swift
  61. log_facility= LOG_LOCAL4
  62. [pipeline:main]
  63. pipeline= account-server
  64. [app:account-server]
  65. use =egg:swift#account
  66. [account-replicator]
  67. vm_test_mode= yes
  68. [account-auditor]
  69. [account-reaper]
  70. 6.  /etc/swift/account-server/4.conf
  71. [DEFAULT]
  72. devices= /srv/4/node
  73. mount_check= false
  74. bind_port= 6042
  75. user =swift
  76. log_facility= LOG_LOCAL5
  77. [pipeline:main]
  78. pipeline= account-server
  79. [app:account-server]
  80. use =egg:swift#account
  81. [account-replicator]
  82. vm_test_mode= yes
  83. [account-auditor]
  84. [account-reaper]
  85. 7.  /etc/swift/container-server/1.conf
  86. [DEFAULT]
  87. devices= /srv/1/node
  88. mount_check= false
  89. bind_port= 6011
  90. user =swift
  91. log_facility= LOG_LOCAL2
  92. [pipeline:main]
  93. pipeline= container-server
  94. [app:container-server]
  95. use =egg:swift#container
  96. [container-replicator]
  97. vm_test_mode= yes
  98. [container-updater]
  99. [container-auditor]
  100. [container-sync]
  101. 8.  /etc/swift/container-server/2.conf
  102. [DEFAULT]
  103. devices= /srv/2/node
  104. mount_check= false
  105. bind_port= 6021
  106. user =swift
  107. log_facility= LOG_LOCAL3
  108. [pipeline:main]
  109. pipeline= container-server
  110. [app:container-server]
  111. use =egg:swift#container
  112. [container-replicator]
  113. vm_test_mode= yes
  114. [container-updater]
  115. [container-auditor]
  116. [container-sync]
  117. 9.  /etc/swift/container-server/3.conf
  118. [DEFAULT]
  119. devices= /srv/3/node
  120. mount_check= false
  121. bind_port= 6031
  122. user =swift
  123. log_facility= LOG_LOCAL4
  124. [pipeline:main]
  125. pipeline= container-server
  126. [app:container-server]
  127. use =egg:swift#container
  128. [container-replicator]
  129. vm_test_mode= yes
  130. [container-updater]
  131. [container-auditor]
  132. [container-sync]
  133. 10./etc/swift/container-server/4.conf
  134. [DEFAULT]
  135. devices= /srv/4/node
  136. mount_check= false
  137. bind_port= 6041
  138. user =swift
  139. log_facility= LOG_LOCAL5
  140. [pipeline:main]
  141. pipeline= container-server
  142. [app:container-server]
  143. use =egg:swift#container
  144. [container-replicator]
  145. vm_test_mode= yes
  146. [container-updater]
  147. [container-auditor]
  148. [container-sync]
  149. 11./etc/swift/object-server/1.conf
  150. [DEFAULT]
  151. devices= /srv/1/node
  152. mount_check= false
  153. bind_port= 6010
  154. user =swift
  155. log_facility= LOG_LOCAL2
  156. [pipeline:main]
  157. pipeline= object-server
  158. [app:object-server]
  159. use =egg:swift#object
  160. [object-replicator]
  161. vm_test_mode= yes
  162. [object-updater]
  163. [object-auditor]
  164. 12./etc/swift/object-server/2.conf
  165. [DEFAULT]
  166. devices= /srv/2/node
  167. mount_check= false
  168. bind_port= 6020
  169. user =swift
  170. log_facility= LOG_LOCAL3
  171. [pipeline:main]
  172. pipeline= object-server
  173. [app:object-server]
  174. use =egg:swift#object
  175. [object-replicator]
  176. vm_test_mode= yes
  177. [object-updater]
  178. [object-auditor]
  179. 13./etc/swift/object-server/3.conf
  180. [DEFAULT]
  181. devices= /srv/3/node
  182. mount_check= false
  183. bind_port= 6030
  184. user =swift
  185. log_facility= LOG_LOCAL4
  186. [pipeline:main]
  187. pipeline= object-server
  188. [app:object-server]
  189. use =egg:swift#object
  190. [object-replicator]
  191. vm_test_mode= yes
  192. [object-updater]
  193. [object-auditor]
  194. 14./etc/swift/object-server/4.conf
  195. [DEFAULT]
  196. devices= /srv/4/node
  197. mount_check= false
  198. bind_port= 6040
  199. user =swift
  200. log_facility= LOG_LOCAL5
  201. [pipeline:main]
  202. pipeline= object-server
  203. [app:object-server]
  204. use =egg:swift#object
  205. [object-replicator]
  206. vm_test_mode= yes
  207. [object-updater]
  208. [object-auditor]
复制代码
11. 创建swift运行脚本
  1. sudo mkdir /home/swift/bin
  2. sudo chown swift:swift -R /home/swift/
  3. 在/etc/profile文件末尾添加如下代码
  4. export PATH=$PATH:/home/swift/bin
复制代码
  1. 1.  创建/home/swift/bin/remakerings
  2. #!/bin/bash
  3. cd  /etc/swift
  4. rm -f*.builder *.ring.gz backups/*.builder backups/*.ring.gz
  5. swift-ring-builder object.builder create 18 3 1
  6. swift-ring-builder container.builder create 18 3 1
  7. swift-ring-builder account.builder create 18 3 1
  8. swift-ring-builder object.builder add z1-127.0.0.1:6010/sdb1 1
  9. swift-ring-builder object.builder add z2-127.0.0.1:6020/sdb2 1
  10. swift-ring-builder object.builder add z3-127.0.0.1:6030/sdb3 1
  11. swift-ring-builder object.builder add z4-127.0.0.1:6040/sdb4 1
  12. swift-ring-builder container.builder add z1-127.0.0.1:6011/sdb1 1
  13. swift-ring-builder container.builder add z2-127.0.0.1:6021/sdb2 1
  14. swift-ring-builder container.builder add z3-127.0.0.1:6031/sdb3 1
  15. swift-ring-builder container.builder add z4-127.0.0.1:6041/sdb4 1
  16. swift-ring-builder account.builder add z1-127.0.0.1:6012/sdb1 1
  17. swift-ring-builder account.builder add z2-127.0.0.1:6022/sdb2 1
  18. swift-ring-builder account.builder add z3-127.0.0.1:6032/sdb3 1
  19. swift-ring-builder account.builder add z4-127.0.0.1:6042/sdb4 1
  20. swift-ring-builder object.builder rebalance
  21. swift-ring-builder container.builder rebalance
  22. swift-ring-builder account.builder rebalance
  23. 2.  创建/home/swift/bin/sfstart
  24. #!/bin/bash
  25. swift-init all start
  26. sudo chown swift:swift -R /var/log/swift/
  27. sudo chmod a+w /var/run/swift/*
  28. 3.  创建/home/swift/bin/sfstop
  29. #!/bin/bash
  30. Swift-init all stop
  31. 4.  创建/home/swift/bin/resetswift
  32. #!/bin/bash
  33. swift-init all stop
  34. find /var/log/swift -type f -exec rm -f {} \;
  35. sudo umount /mnt/sdb1
  36. sudo mkfs.xfs -f -i size=1024 /srv/swift-disk
  37. sudo chmod a+w /srv/swift-disk
  38. sudo mount /mnt/sdb1
  39. sudo rm -rf /srv/1 /srv/2 /srv/3 /srv/4
  40. sudo mkdir /mnt/sdb1/1 /mnt/sdb1/2 /mnt/sdb1/3 /mnt/sdb1/4
  41. sudo chown swift:swift /mnt/sdb1/*
  42. for x in {1..4}; do sudo ln -s /mnt/sdb1/$x/srv/$x; done
  43. sudo mkdir -p /srv/1/node/sdb1 /srv/2/node/sdb2 /srv/3/node/sdb3/srv/4/node/sdb4
  44. sudo chown swift:swift -R /srv/*
  45. sudo rm -f /var/log/debug /var/log/messages /var/log/rsyncd.log/var/log/syslog
  46. sudo chown -R swift:swift /mnt/sdb1/*
  47. sudo service rsyslog restart
  48. sudo service memcached restart
  49. 在/home/swift/bin/resetswift中,如果你使用的是单独分区来做存储点的,你需要把sudo mkfs.xfs -f -i size=1024 /srv/swift-disk和sudo chmod a+w /srv/swift-disk中的/srv/swift-disk改为该分区的设备名,例如本文第七步第一小点所说的那样,在这里我这里应该改为/dev/sdb1,具体情况根据你的系统而定
  50. 5.  使/home/swift/bin下的文件具有可执行权限
  51. sudo  chmod a+x /home/swift/bin/*
复制代码


12. 生成ring文件
source  /etc/profile  #下次启动系统是这条命令就不用执行了
remakerings     #执行该命令在/etc/swift中生成相应的文件

13. 启动swift服务
source  /etc/profile  #下次启动系统是这条命令就不用执行了
sfstart       #执行这个需要一段时间
#当启动的时候可能会报WARNING: Unable to increase filedescriptor limit.  Running as non-root?  这是正常情况

14. 测试swift服务
1.  初始化swauth
swauth-prep  -K swauthkey     #执行成功的话则不会有任何信息显示
#在这一步经常会出现Auth subsystem prep failed: 500 ServerError这种错误,发生错误的原因可能是你的/srv下的目录和/mnt/sdb1下的目录的拥有者不是属于swift用户。
2.  添加用户
swauth-add-user  -A  http://127.0.0.1:8080/auth/ -K swauthkey -a test tester testing   #在tenant test中添加tester用户并设置密码为testing

3.  检测swift运行状态
swift  -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing stat  -v       #运行成功则会返回类似如下的信息

AuthToken: AUTH_tkf5b6840d9ce64d7298a800505a2ed857
Account:AUTH_3f7fb037-57b6-4771-831a-8fd811bcc3c7
Containers:0
Objects:0
Bytes:0
Accept-Ranges:bytes
X-Trans-Id:txdb39ed113c1c49299ebb364a4246a3a9
注:红色字体部分是根据具体情况而定的,并不是每个人系统返回都是一样的,假如出现了错误你可以在/var/log/swift/目录下查询到相应的信息
15. 上传和下载文件
1.  获取X-Storage-Token和X-Storage-Url
curl  -v -H 'X-Storage-User: test:tester' -H 'X-Storage-Pass: testing'  http://127.0.0.1:8080/auth/v1.0        #执行完该条语句会返回相应的信息,例如(返回信息的一小部分):
HTTP/1.1200 OK
X-Storage-Token:AUTH_tkf5b6840d9ce64d7298a800505a2ed857
X-Auth-Token:AUTH_tkf5b6840d9ce64d7298a800505a2ed857
swift  -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list       #查看test用户的container列表
3.  上传和列出文件
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing upload cynric/etc/swift/proxy-server.conf #将/etc/swift/proxy-server.conf文件上传到cynric容器中

swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing list cynric   #列出cynric容器中的文件
4.  下载文件
swift -A http://127.0.0.1:8080/auth/v1.0 -U test:tester -K testing download cynric   #将cynric容器下的所有文件下载到本地
注意:如果上传文件的时候,有嵌套目录,则上传到swift上后,也是以嵌套目录的形式存在,下载后,会在当前目录创建一样的嵌套目录。




没找到任何评论,期待你打破沉寂

您需要登录后才可以回帖 登录 | 立即注册

本版积分规则

关闭

推荐上一条 /2 下一条