Concerning k8s use cases …
I will be heading an initiative to migrate our (PeopleDataLabs) processing substrate from traditional spark+yarn+hadoop architecture to a native spark+k8s configuration.
Info on how to do this is sparse ATM as the k8s driver was just recently released in spark 2.3, but I have managed to get a native spark+k8s strawman working on CDK! Now to make it accommodate our workload
Find a reasonably current (at time of writing), high level update on this bleeding edge - https://databricks.com/blog/2018/09/26/whats-new-for-apache-spark-on-kubernetes-in-the-upcoming-apache-spark-2-4-release.html