Handy MongoDB queries for debugging


#1

Often when you’re trying to chase down a bug it can be a bit of a nightmare to find the query syntax you need to get the right information. When you have a useful one it’s probably worth adding it here.

Finding aborted transactions (“state changing too fast” errors)

This error generally indicates a programming error rather than a real contention issue - it happens when there are assertions in the []txn.Op slice that aren’t being checked when constructing the slice.

Find the last aborted transaction that wasn’t touching the link layer devices collection. (We were trying to track down a bug in upgrade charm handling but were stymied by the aborted txns from a different bug.)

db.txns.find(
    {s:5, "o.c":{$ne:"linklayerdevices"}}
).sort({_id:-1}).limit(1).pretty()

Transaction fields:

  • s is the transaction state - 5 is aborted.
  • o is an array of operations (insert/update/delete/assert)
  • c is the collection the operation works on.

Generally once you have the failed transaction you can go through the assertions in the transaction one by one until you find the failing one. Then you can go back to the code that generated the transaction and see why it didn’t check that condition in its buildTxn.


#2

Another one I started using after @jameinel wonderful analysis on bug LP: #1797816 is:

db.getCollectionNames().forEach(
    function(aRow) {
        print("--- " + aRow + " ---");
        db.getCollection(aRow).aggregate([{$match: {"txn-queue.100": {$exists: 1}}}, 
                                          {$project: {"_id": 1,  len: {$size: "$txn-queue"}}}]).shellPrint(); 
    });

This one will iterate over all the collections and print the id of documents that have a txn-queue larger than 100 elements.