Selenium grid on Kubernetes

Selenium Hub Overlearning

You can skip this section or try it for extra credit.

We may use kubectl exec for container introspection:

kubectl exec selenium-hub-3216163580-7pqtx -- ps aux

As you can see we have a java process running

kubectl exec selenium-hub-3216163580-7pqtx -- ps aux
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
seluser      1  0.0  0.1  18044  2688 ?        Ss   17:20   0:00 /bin/bash /opt/bin/entry_point.sh
seluser      7  0.1  3.1 2928868 64864 ?       Sl   17:20   0:13 java -jar /opt/selenium/selenium-server-standalone.jar -role hub -hubConfig /opt/selenium/config.json
seluser     87  0.0  0.1  34424  2856 ?        Rs   20:54   0:00 ps aux

We may also look at the config file and test the service internally:

# inspect the selenium config file.
kubectl exec selenium-hub-3216163580-7pqtx -- cat /opt/selenium/config.json

# see if selenium is really listening on port 4444.
kubectl exec selenium-hub-3216163580-7pqtx -- wget 127.0.0.1:4444 -O -

You can even shell into the container:

kubectl exec -it elenium-grid-3216163580-7pqtx -- /bin/bash

Exit out of the container and lets setup some Selenium Nodes!

Selenium Node Overlearning

You can skip this section or try it for extra credit.

The first time I tried to launch a Selenium node and I had trouble.

I ran this:

kubectl get pods
kubectl run selenium-node-chrome --image selenium/node-chrome:2.53.1
kubectl get pods

The new pod went into status CrashLoopBackOff:

NAME                                    READY     STATUS             RESTARTS   AGE
selenium-grid-3216163580-7pqtx          1/1       Running            1          3d
selenium-node-chrome-4019562870-mcpfg   0/1       CrashLoopBackOff   6          6m

To troubleshoot, I used the following commands:

kubectl describe pod selenium-node-chrome

This command allowed me to review the Kubernetes level logs. Everything seemed healthy so next I looked at the Docker level logs:

kubectl logs selenium-node-chrome-4019562870-mcpfg
Not linked with a running Hub container

Ok, the error Not linked with a running Hub container appears when Selenium node cannot find the hub.

Docker has a --link flag to link containers together, Kubernetes doesn't have this. After some research, it seems --link manages ENV vars.

You can see the environment vars of a pod using this command:

kubectl exec selenium-hub-3216163580-7pqtx -- printenv

I learned that the selinum-node-chrome docker image expects some ENV vars and if it doesn't get them, it goes into a crash loop.

I reached out over IRC in the #kubernetes and #selenium channels to ask about the ENV vars needed. A really helpful user named smccarthy linked me to this:

https://github.com/kubernetes/kubernetes/tree/master/examples/selenium

Apparently one of the example Kubernetes clusters is a Selenium Grid setup!

Looking over the example, I found the ENV vars that the selenium-node containers expect: HUB_PORT_4444_TCP_ADDR and HUB_PORT_4444_TCP_PORT

Man, why would they put the port (4444) in the key?

Anyways, we pass these key/values when creating the container like this:

kubectl run selenium-node-chrome --image selenium/node-chrome:2.53.1 --env="HUB_PORT_4444_TCP_ADDR=selenium-hub" --env="HUB_PORT_4444_TCP_PORT=4444"