Docker 遇到 device-mapper problem、cookie dm_task_set_cookie failed

Docker 遇到無法啟動，於 /var/log/syslog 看到下述訊息：

• msg="devmapper: Usage of loopback devices is strongly discouraged for production use. Please use `--storage-opt dm.thinpooldev` or use `man docker` to refer to dm.thinpooldev section."
• msg="[graphdriver] prior storage driver devicemapper failed: devmapper: Base Device UUID and Filesystem verification failed: devicemapper: Can't set cookie dm_task_setcookie failed"
• Error starting daemon: error initializing graphdriver: devmapper: Base Device UUID and Filesystem verification failed: devicemapper: Can't set cookie dm_task_set_cookie failed
• docker.service: Main process exited, code=exited, status=1/FAILURE
• Failed to start Docker Application Container Engine.

要怎麼在不重新開機的情況讓 Docker 啟動呢？

Docker 遇到 device-mapper problem、cookie dm_task_set_cookie failed

由錯誤訊息查看，主要錯誤訊息在這邊：

• prior storage driver devicemapper failed: devmapper: Base Device UUID and Filesystem verification failed: devicemapper: Can't set cookie dm_task_set_cookie failed"

由此篇文章 Unable to start Docker due device-mapper problem "Can't set cookie dm_task_set_cookie failed" 想要將 device 先 umount 掉，也是失敗，如下述：

1. for i in /dev/mapper/docker-; do umount $i; dmsetup remove $i; done # 失敗
2. mv /var/lib/docker/ /var/lib/docker.backup # 會自動建立 /var/lib/docker，但是一樣無法啟動
3. rm -fr /var/lib/docker # 還原
4. mv /var/lib/docker.backup/ /var/lib/docker # 還原

所以 device-mapper 不是主要問題，從下述錯誤訊息的角度來查：

• Unable to start Docker due device-mapper problem "Can't set cookie dm_task_set_cookie failed"

查到此篇文章：devicemapper: Can't set cookie dm_task_set_cookie failed

註：以下都是用 root 執行，用 $ 開頭是命令列

1. $ ipcs # 超過 128個 (wc -l)
   ------ Semaphore Arrays --------
   key semid owner perms nsems
   0x0d4d3358 238977024 root 600 1
   0x0d4d0ec9 270172161 root 600 1
   0x0d4dc02e 281640962 root 600 1
   0x0d4db8d2 291045379 root 600 1
2. $ cat /proc/sys/kernel/sem # 查看 semaphore 的設定值 250 32000 32 128
3. $ echo 250 32000 32 1024 > /proc/sys/kernel/sem # 先給比較大的值
4. $ sudo systemctl restart docker.service # 就可以啟動了，應該是 semaphore leaks
5. $ echo 250 32000 32 128 > /proc/sys/kernel/sem # 還原回原始設定
6. $ echo 32000 1024000000 500 32000 > /proc/sys/kernel/sem # 還原回原始設定
7. $ sudo systemctl stop docker.service # 先關掉
8. $ dmsetup udevcomplete_all # 清掉所有 leaks，重點在這行
9. $ ipcs # Semaphore Arrays 都清空了
10. $ sudo systemctl start docker.service # 就可以啟動了

相關網頁

• Semaphore (programming) - Wikipedia
• How to get proccesses currently running semaphores by /proc?
• device-mapper: remove ioctl on luks-xxxx failed: Device or resource busy