此处将为大家介绍关于树莓派4Braspberrypi节点加入kubeedge集群并解决error:FollowingCgroupsubsystemnotmounted:[memory]的详细内容，此外

此处将为大家介绍关于树莓派 4B raspberrypi 节点加入 kubeedge 集群并解决 error: Following Cgroup subsystem not mounted: [memory]的详细内容，此外，我们还将为您介绍关于64-Bit Server VM warning: INFO: os::commit_memory(0x) failed; error=''Cannot allocate memory ？？？、ASM problem : ORA-15001: diskgroup "DGROUP1" does not exist or is not mounted ORA-15040: diskgroup is incomplete、Build Yocto system for Raspberry Pi under win10-x86、Error getting ConfigMap kube-system:kube-dns err: configmaps “kube-dns“ not found的有用信息。

• 树莓派 4B raspberrypi 节点加入 kubeedge 集群并解决 error: Following Cgroup subsystem not mounted: [memory]
• 64-Bit Server VM warning: INFO: os::commit_memory(0x) failed; error=''Cannot allocate memory ？？？
• ASM problem : ORA-15001: diskgroup "DGROUP1" does not exist or is not mounted ORA-15040: diskgroup is incomplete
• Build Yocto system for Raspberry Pi under win10-x86
• Error getting ConfigMap kube-system:kube-dns err: configmaps “kube-dns“ not found

树莓派 4B raspberrypi 节点加入 kubeedge 集群并解决 error: Following Cgroup subsystem not mounted: [memory]

树莓派 raspberrypi 上先安装好 Docker

树莓派 4B raspberrypi 离线安装 Docker

树莓派 4B raspberrypi 在线安装 Docker

华为云 arm 架构轻松安装 kubeedge

树莓派 raspberrypi 当边缘节点在线加入 kubeedge 集群

kubeedge 官网下载 keadm: https://github.com/kubeedge/kubeedge/releases

树莓派是 arm 架构的，注意不是 arm64 位，下载包: keadm-v1.5.0-linux-arm.tar.gz

#查看树莓派内核版本
uname -r 
Linux raspberrypi 5.10.17-v7l+ #1403 SMP Mon Feb 22 11:33:35 GMT 2021 armv7l GNU/Linux
#或者使用 uname -a

#在cloudcore主机上,即k8s master节点, 获取kubeedge的token, 执行命令获取token
./keadm-v1.6.1-linux-arm64/keadm/keadm gettoken

# 树莓派上执行,解压
tar -zxvf keadm-v1.5.0-linux-arm.tar.gz 

#加入kubeedge集群管理
./keadm-v1.5.0-linux-arm/keadm/keadm join --cloudcore-ipport=192.168.0.5:10000 --cgroupdriver=systemd --edgenode-name=raspberrypi --kubeedge-version=1.5.0 --token=3cca11140cab1c229d9cfcb9fec5b56fc46d8a61d5607e6d28a1df949d228136.eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJleHAiOjE2MTI0MjMxNzF9.lW1VhW-II45HZ9Hx3cROi2Q2zp8MNEcjmgh1I4jZZY0

#需要等待, 它会在线安装mosquitto MQTT, 然后从github下载kubeedge的包,访问github比较慢,后面的博客我会讲到离线部署kubeedge

--cloudcore-ipport=192.168.0.5:10000 cloudcore 主机 IP 和端口

--cgroupdriver=systemd 使用 systemd 驱动，在 /etc/docker/daemon.json 配置文件中添加: "exec-opts": ["native.cgroupdriver=systemd"]

--edgenode-name=raspberrypi 节点名称

--kubeedge-version=1.5.0 kubeedge 版本

--token 在 cloudcore 主机上获取的 token

解决 error: system validation failed - Following Cgroup subsystem not mounted: [memory]

#上面加入kubeedge集群管理, 报错:
 Failed to start container manager, err: system validation failed - Following Cgroup subsystem not mounted: [memory]
E0125 16:42:09.131414    1655 edged.go:291] initialize module error: system validation failed - Following Cgroup subsystem not mounted: [memory]

#解决问题
#修改/boot/cmdline.txt
sudo vim /boot/cmdline.txt
cgroup_enable=memory cgroup_memory=1
添加在同一行的最后面,接着内容后空格后添加, 注意:不要换行添加

#重启机器配置生效
reboot

配置 edgecore 开机自启动服务

#查看edgecore启动情况
ps aux|grep edgecore

#查看启动状态
sudo systemctl status edgecore

#查看docker开机启动状态 enabled:开启, disabled:关闭
systemctl is-enabled edgecore

#设置开机自启动
systemctl enable edgecore

64-Bit Server VM warning: INFO: os::commit_memory(0x) failed; error=''Cannot allocate memory ？？？

服务器异常了。。。。Java HotSpot(TM) 64-Bit Server VM warning: INFO: os::commit_memory(0x00000005cc400000, 349700096, 0) failed; error=''Cannot allocate memory'' (errno=12)
#
# There is insufficient memory for the Java Runtime Environment to continue.
# Native memory allocation (mmap) failed to map 349700096 bytes for committing reserved memory.
# An error report file with more information is saved as:
# /root/hs_err_pid18533.log

ASM problem : ORA-15001: diskgroup "DGROUP1" does not exist or is not mounted ORA-15040: diskgroup is incomplete

Build Yocto system for Raspberry Pi under win10-x86

build Yocto system for Raspberry Pi under win10-x86

Preparation

1. Reference:

   • Building Our First Poky Image for the Raspberry Pi
   • Yocto Project official site

2. platform: Windows 10.

   • host machine: Windows 10, x86
   • target machine: Raspberry Pi3 B+, ARM

Set up development environment for host machine

1. Download Docker for cross-compilation environment

Concerning the host machine, naive Linux host is easier to start(See Quick Start Guide for Systems Developers). For Windows and Mac OS, we leverage docker to set up cross-compilation development environment(look at CROPS). This exmaple demonstrates solution for windows10.

Choose the right version of docker concerning your PC platform.

For Win10, we choose Docker CE Stable. Download docker and install it.

2. Configurate docker-machine and create poky container

See instructions and do the following steps.

2.1 Change default vm settings

• Run powershell as administrator and run the following commands.

• Remove the default docker-machine

  docker-machine rm default

• Re-create the default docker-machine

  Win10 has inner hyper-v vitual machine driver, so we don''t need to use the virtualbox in the reference. Enable hyper-v and prepare hyper-v following this instructions.

  • Choose the number of cpus with --hyperv-cpu-count. For this example we''ll use two.
  • Choose the amount of RAM: --hyperv-memory. This is also based on the host hardware. However, choose at least 4GB.
  • Choose the amount of disk space: --hyperv-disk-size. It is recommended that this be at least 50GB since building generates a lot of output. In this example we''ll choose 50GB.
  • Create vm with new settings

  docker-machine create -d hyperv --hyperv-cpu-count=2 --hyperv-memory=4096 --hyperv-disk-size=50000 default

  • Restart docker

Restart docker-machine to avoid some unexpected errors later on.
```bash
docker-machine restart
exit
```
Use command `docker-machine ls` to list and check your settings.

2.2 Create the samba container

• Useful docker command:

  docker image ls // list all images
  docker container ls // list all running containers
  docker container ls --all // list all containers
  docker container stop [NAME or ID] // stop certain container
  docker container rm [NAME or ID] // remove certain container(container should be stopped first before removed)

See more in Docker tutorial.

• Create a volume. A volume is used to persistent data on the host machine.

  docker volume create --name myvolume
  docker run -it --rm -v myvolume:/workdir busybox chown -R 1000:1000 /workdir

The second command is executed to change the read-write permission of the /workdir.

• create container samba to monitor poky container.

  docker create -t -p 445:445 --name samba -v myvolume:/workdir crops/samba
  docker start samba

If you start samba succefully, run docker-machine ip to get ip(something like 192.168.99.100), and then to see the workdir open the file browser in windows (win+e) and type \\192.168.99.100\workdir.
If you fail to start samba and get error message like:

Error response from daemon: driver failed programming external connectivity on endpoint samba (43a4e1ec467d6f584d5799e7ed3b333bd2bcabc17f0086ec0851024b018620bc): Error starting userland proxy: Bind for 0.0.0.0:445: unexpected error Permission denied
Your local port 445 is likely occupied by other process. Simply change -p 445:445 to -p 446:445.

docker create -t -p 446:445 --name samba -v myvolume:/workdir crops/samba
docker start samba

2.3 Create a Poky container

Before using the poky container, make sure the samba container is running. Note that if you have started it in a previous terminal it will still be running. Run poky container as follows, note that we use the volume created above when specifying the workdir.

docker run --rm -it -v myvolume:/workdir crops/poky --workdir=/workdir

You will see a prompt that looks like

pokyuser@892e5d2574d6:/workdir$

Build Yocto for your raspberry pi

1. Go to Raspberry Pi Official site to check the board model number.

   Choose the model number for your certain hardware. For Example, we have a Raspberry Pi 3B+ board. Thus the hardware in raspberry pi is BCM2837B0. We select raspberrypi3 as the choice for MACHINE. See the conf file in meta-raspberrypi/conf/machine for details. Remenber this machine type, later you will need to fill it in the local.conf file.

   Supported machine:

   • raspberrypi (BCM2835)
   • raspberrypi0 (BCM2835)
   • raspberrypi0-wifi (BCM2835)
   • raspberrypi2 (BCM2836 or BCM2837 v1.2+)
   • raspberrypi3 (BCM2837)
   • raspberrypi3-64 (64 bit kernel & userspace)
   • raspberrypi-cm (dummy alias for raspberrypi) (BCM2835)
   • raspberrypi-cm3 (dummy alias for raspberrypi2) (BCM2837)

Note: The raspberrypi3 machines include support for Raspberry Pi 3B+.

The example in this article use Yocto Project version 2.6 and poky version 2.6

1. Download the Poky metadata

   git clone git://git.yoctoproject.org/poky (branch master)

2. Download the Raspberry Pi BSP and dependencies metadata as layers

   cd poky
   git clone git://git.yoctoproject.org/meta-raspberrypi

   Poky is just a reference minimal system for us to start, we can add layers to customize our linux system specific to our hardware. Here we have a raspberry pi board. So we need the official BSP: meta-raspberrypi layer. What''s more, according to Yocto: meta-raspberrypi, there are other dependencies we need:

   • URI: git://git.openembedded.org/meta-openembedded

     • layers: meta-oe, meta-multimedia, meta-networking, meta-python
     • branch: master
     • revision: HEAD

git clone git://git.openembedded.org/meta-openembedded

Now we have `/meta-raspberrypi` and `/meta-openembedded` folders under `/poky` directory.

1. The oe-init-build-env script

   The Poky directory contains a script named oe-init-build-env. This is a script for the configuration/initialization of the build environment. It is not intended to be executed but must be "sourced". Its work, among others, is to initialize a certain number of environment variables and place yourself in the build directory’s designated argument. The script must be run as shown here:

   source oe-init-build-env [build-directory]

   Here, build-directory is an optional parameter for the name of the directory where the environment is set (for example, we can use several build directories in a single Poky source tree); in case it is not given, it defaults to build. The build-directory folder is the place where we perform the builds. But, in order to standardize the steps, we will use the following command throughout to initialize our environment:

   $ source oe-init-build-env rpi-build
   
   ### Shell environment set up for builds. ###
   
   You can now run ''bitbake ''
   Common targets are:
   core-image-minimal
   core-image-sato
   meta-toolchain
   adt-installer
   meta-ide-support
   You can also run generated qemu images with a command like ''runqemu qemux86''

   When we initialize a build environment, it creates a directory (the conf directory) inside rpi-build. This folder contains two important files:

   • local.conf: It contains parameters to configure BitBake behavior.
   • bblayers.conf: It lists the different layers that BitBake takes into account in its implementation. This list is assigned to the BBLAYERS variable.

2. Editing the local.conf file

   Search in the /rpi-build/conf/local.conf file. Find the MACHINE ??= line and change it to :

   MACHINE ??= "raspberrypi3"

   Here, the "raspberrypi3" is the machine type we found out before according to our board model number.

3. Editing the bblayers.conf file

   Add the meta-raspberrypi layer and meta-oe, meta-multimedia, meta-networking, meta-python dependences layers to the bblayers.conf file. The file should look like this:

   # POKY_BBLAYERS_CONF_VERSION is increased each time build/conf/bblayers.conf
   # changes incompatibly
   POKY_BBLAYERS_CONF_VERSION = "2"
   BBPATH = "${TOPDIR}"
   
   BBFILES ?= ""
   
   BBLAYERS ?= " \
   /workdir/dilin/poky/meta \
   /workdir/dilin/poky/meta-poky \
   /workdir/dilin/poky/meta-yocto-bsp \
   /workdir/dilin/poky/meta-raspberrypi \
   /workdir/dilin/poky/meta-openembedded/meta-oe \
   /workdir/dilin/poky/meta-openembedded/meta-multimedia \
   /workdir/dilin/poky/meta-openembedded/meta-networking \
   /workdir/dilin/poky/meta-openembedded/meta-python \
   "

4. Building the Poky image

   • Choosing the image

At this stage, we will have to look at the available images as to whether they are compatible with our platform (.bb files).
Poky provides several predesigned image recipes that we can use to build our own binary image. We can check the list of available images by running the following command from the poky directory:

 ```bash
 $ ls meta*/recipes*/images/*.bb
 ```
We want to build some raspberrypi-specific image. There are there of them: `rpi-test-image`, `rpi-hwup-image` and `rpi-basic-image`. According to [meta-raspberrypi docs](https://meta-raspberrypi.readthedocs.io/en/latest/layer-contents.html), the latter two images have been deprecated, so we build the `rpi-test-image` image. It is based on `core-image-base` which includes most of the packages in this layer and some media samples.
* Running BitBake
Under our build directory `rpi-build`, run the following command:

```bash
pokyuser@49065a978b08:/workdir/poky/rpi-build$ bitbake rpi-test-image
Loading cache: 100% |#########################################################################################################################################################################################################################################################################################| Time: 0:00:00
Loaded 3164 entries from dependency cache.
NOTE: Resolving any missing task queue dependencies

Build Configuration:
BB_VERSION           = "1.40.0"
BUILD_SYS            = "x86_64-linux"
NATIVELSBSTRING      = "universal"
TARGET_SYS           = "arm-poky-linux-gnueabi"
MACHINE              = "raspberrypi3"
DISTRO               = "poky"
DISTRO_VERSION       = "2.6+snapshot-20181203"
TUNE_FEATURES        = "arm armv7ve vfp thumb neon vfpv4 callconvention-hard cortexa7"
TARGET_FPU           = "hard"
meta
meta-poky
meta-yocto-bsp       = "master:7faf6a00ba55db5cb5f2c21a72d09df8d784e9e8"
meta-raspberrypi     = "master:c8a05f2fcf17ca2556f8a56086ea36da2e777d0f"
meta-oe
meta-multimedia
meta-networking
meta-python          = "master:ebc7b9e20ac22f6f2ad373621917f53e8a9af81c"

Initialising tasks: 100% |####################################################################################################################################################################################################################################################################################| Time: 0:00:03
Sstate summary: Wanted 187 Found 186 Missed 1 Current 1175 (99% match, 99% complete)
NOTE: Executing SetScene Tasks
NOTE: Executing RunQueue Tasks
NOTE: Tasks Summary: Attempted 4006 tasks of which 4006 didn''t need to be rerun and all succeeded.
```

If encountered with the below error message, add a line `LICENSE_FLAGS_WHITELIST = "commercial"` in *rpi-build/conf/local.conf* file.
> ERROR: Nothing PROVIDES ''libav'' (but /workdir/poky/meta-raspberrypi/recipes-multimedia/omxplayer/omxplayer_git.bb DEPENDS on or otherwise requires it)ffmpeg PROVIDES libav but was skipped: because it has a restricted license ''commercial''. Which is not whitelisted in LICENSE_FLAGS_WHITELIST

The first bitbake can take quite a long time. Elapsed time varies concerning the layers you add. For reference, it takes 4h for the first bitbake of `core-image-minimal` image. And over 8h to bitbake `rpi-test-image` image.

Flash image to sdcard for raspberry pi

1. Copy image from docker to your PC

   After hours of baking, we can rejoice with the result and the creation of the system image for our target:

   $ ls rpi-build/tmp/deploy/images/raspberrypi3/*sdimg
   rpi-test-image-raspberrypi3-20181203014212.rootfs.rpi-sdimg  rpi-test-image-raspberrypi3.rpi-sdimg

   rpi-test-image-raspberrypi3-20181203014212.rootfs.rpi-sdimg can be used with Win32DiskImager in Windows, while in Linux you can use the dd command with the rpi-test-image-raspberrypi3.rpi-sdimg directly to flash image to SD card.

   Open another powershell terminal and run:

   docker cp 6446fd34ba5e:/workdir/poky/rpi-build/tmp/deploy/images/raspberrypi3/rpi-test-image-raspberrypi3-20181203014212.rootfs.rpi-sdimg [directory in your windows] // Replace 6446fd34ba5e with your samba container ID

2. Flash image to SD card

   See Raspberry pi doc.

• Format SD card first

  Plug in your SD card and you can see two disk device concerning your SD card. One is named boot, another is USB drive. Download SD Format and format the boot device.

  Format SD-card before flashing. Or you propabably will get a "Error 5 access denied error" when trying to write an image to your SD-card.

• Flash image to SD card

  Download Win32DiskImager(See raspberry pi instructions). From Win32DiskImager choose the rpi-test-image-raspberrypi3-20181203014212.rootfs.rpi-sdimg from the directory you have just saved it, and choose the boot disk device.
  Carefully check you choose the right device and click write.

1. After you successfully start the system on raspberry pi, you see:

   raspberrypi3 login:

   Input root and you can use the system now.

   HURRAY!!!

Troubleshooting

• If you cannot bind local 445 port to samba''s 445 port, you can use the below command to validate everything goes well in pocky. Open another powershell terminal as administrator and run the below docker command to help debug.

PS C:\windows\system32> docker exect -it [your samba container ID] bash // go into samba container
root@a6fd96ce2b55:# cd workdir/

• When BitBake fails for some package uninstalled, open another powershell terminal as administrator and run the below docker command to help debug.

PS C:\windows\system32> docker exec -it -u 0 [your poky container ID] bash
root@a6fd96ce2b55:# sudo apt-get install [your needed package name]

• Open another powershell terminal as administrator and run the below docker command to check image''s history actions.

docker history crops/poky

• If you encounter BitBake error with messages showed below, delete the /rpi-build/tmp folder to free some space. Run command du -sh ./tmp under /rpi-build directory to see the size of tmp folder. Or you can open another powershell terminal as administrator and run the following command to check the size of the build directory.

docker exec -it samba bash
cd workdir/poky/rpi-build
du-h --max-depth=1

• If you plug in your sd-card which has been succefully wrote the rpi-test-image, and the system boots continuously prints out:

init: id "s0" respawning too fast: disabled for 5 minutes

Go back and add ENABLE_UART = "1" in local.conf. Re-bake your image and the problem can be solved.

Error getting ConfigMap kube-system:kube-dns err: configmaps “kube-dns“ not found

[root@k8s-master ~]# kubectl exec -it busyBoxx sh
/ # nslookup Nginx
Server:    10.254.230.254
Address 1: 10.254.230.254

nslookup: can't resolve 'Nginx'
/ # cat /etc/resolv.conf 
search default.svc.cluster.local svc.cluster.local cluster.local
nameserver 10.254.230.254
nameserver 8.8.8.8
options ndots:5

[root@k8s-master ~]# kubectl logs -f kube-dns-3204099596-x7vdj -c kubedns -n kube-system
I0613 20:15:12.651122       1 dns.go:42] version: v1.6.0-alpha.0.680+3872cb93abf948-dirty
I0613 20:15:12.651262       1 server.go:107] Using http://192.168.150.61:8080 for kubernetes master, kubernetes API: v1
I0613 20:15:12.651550       1 server.go:68] Using configuration read from ConfigMap: kube-system:kube-dns
I0613 20:15:12.651599       1 server.go:113] FLAG: --alsologtostderr="false"
I0613 20:15:12.651613       1 server.go:113] FLAG: --config-map="kube-dns"
I0613 20:15:12.651620       1 server.go:113] FLAG: --config-map-namespace="kube-system"
I0613 20:15:12.651626       1 server.go:113] FLAG: --dns-bind-address="0.0.0.0"
I0613 20:15:12.651630       1 server.go:113] FLAG: --dns-port="10053"
I0613 20:15:12.651637       1 server.go:113] FLAG: --domain="cluster.local."
I0613 20:15:12.651645       1 server.go:113] FLAG: --federations=""
I0613 20:15:12.651651       1 server.go:113] FLAG: --healthz-port="8081"
I0613 20:15:12.651656       1 server.go:113] FLAG: --kube-master-url="http://192.168.150.61:8080"
I0613 20:15:12.651662       1 server.go:113] FLAG: --kubecfg-file=""
I0613 20:15:12.651666       1 server.go:113] FLAG: --log-backtrace-at=":0"
I0613 20:15:12.651673       1 server.go:113] FLAG: --log-dir=""
I0613 20:15:12.651679       1 server.go:113] FLAG: --log-flush-frequency="5s"
I0613 20:15:12.651685       1 server.go:113] FLAG: --logtostderr="true"
I0613 20:15:12.651691       1 server.go:113] FLAG: --stderrthreshold="2"
I0613 20:15:12.651695       1 server.go:113] FLAG: --v="0"
I0613 20:15:12.651700       1 server.go:113] FLAG: --version="false"
I0613 20:15:12.651707       1 server.go:113] FLAG: --vmodule=""
I0613 20:15:12.651756       1 server.go:155] Starting SkyDNS server (0.0.0.0:10053)
I0613 20:15:12.666762       1 server.go:165] Skydns metrics enabled (/metrics:10055)
I0613 20:15:12.669465       1 logs.go:41] skydns: ready for queries on cluster.local. for tcp://0.0.0.0:10053 [rcache 0]
I0613 20:15:12.669518       1 logs.go:41] skydns: ready for queries on cluster.local. for udp://0.0.0.0:10053 [rcache 0]
E0613 20:15:12.679738       1 sync.go:105] Error getting ConfigMap kube-system:kube-dns err: configmaps "kube-dns" not found
E0613 20:15:12.679763       1 dns.go:190] Error getting initial ConfigMap: configmaps "kube-dns" not found, starting with default values
I0613 20:15:12.684971       1 server.go:126] Setting up Healthz Handler (/readiness)
I0613 20:15:12.685000       1 server.go:131] Setting up cache handler (/cache)
I0613 20:15:12.685007       1 server.go:120] Status HTTP port 8081

[root@k8s-master ~]# cat kube-dns-cm.yml 
apiVersion: v1
kind: ConfigMap
Metadata:
  name: kube-dns
  namespace: kube-system
data:
  upstreamNameservers: |
     ["1.2.3.4"]

kubectl create -f kube-dns-cm.yml 

[root@k8s-master ~]# kubectl get deploy,po,svc,cm,ep -n kube-system
NAME              DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
deploy/kube-dns   1         1         1            1           33m

NAME                           READY     STATUS    RESTARTS   AGE
po/kube-dns-3204099596-n2w84   4/4       UnkNown   0          33m
po/kube-dns-3204099596-x7vdj   4/4       Running   0          23m

NAME           CLUSTER-IP       EXTERNAL-IP   PORT(S)         AGE
svc/kube-dns   10.254.230.254   <none>        53/UDP,53/TCP   31m

NAME          DATA      AGE
cm/kube-dns   1         13m

NAME                         ENDPOINTS                   AGE
ep/kube-controller-manager   <none>                      25m
ep/kube-dns                  10.0.55.7:53,10.0.55.7:53   31m
ep/kube-scheduler            <none>                      25m

[root@k8s-master ~]# kubectl exec -it busyBoxx sh
/ # nslookup kubernetes
Server:    10.254.230.254
Address 1: 10.254.230.254 kube-dns.kube-system.svc.cluster.local

Name:      kubernetes
Address 1: 10.254.0.1 kubernetes.default.svc.cluster.local
/ # nslookup Nginx
Server:    10.254.230.254
Address 1: 10.254.230.254 kube-dns.kube-system.svc.cluster.local

Name:      Nginx
Address 1: 10.254.213.70 Nginx.default.svc.cluster.local