In an ideal world we’d have a well-configured centralised monitoring solution in place for every deployment. Unfortunately, some times we don’t and we need to do the best we can with what we have at our disposal.
Without OpsCenter, an ELK stack, or some other centralised monitoring in place, the Cassandra swiss-army knife, nodetool, is what one would use to monitor streaming (bootstrapping, repairing, rebuilding) progress. I personally find the output a bit too verbose and difficult to grok at a glance.
I like to use the following command to get an ongoing, easy to grok picture of streaming progress.
watch -n 10 'nodetool netstats | grep "Receiving\|Sending" | gawk {'"'"' print $1" - "$11/$4*100"% Complete, "($4-$11)/1024/1024/1024" GB remaining" '"'"'}'Here is an example of the output from a node being rebuilt from 4 other nodes.
Here's how it looks on the sending end(s).
Again, this is far from the ideal of a centralised monitoring system. But my hope as this will make life easier (or at least prettier) for somebody working in an environment under constraints that do not make centralised monitoring possible.
