Place the catfish pieces into a mixing bowl and sprinkle with thyme leaves. Toss until the catfish is evenly coated with the thyme. Mix in the mustard with your hands. Cover, and refrigerate overnight, up to 24 hours.
Heat oil in a deep-fryer or large saucepan to 350 degrees F (175 degrees C). Whisk the flour, salt, black pepper, and baking soda together in a mixing bowl.
Dip the catfish pieces one at a time into the flour mixture and shake off the excess. Fry the catfish nuggets in batches until they are no longer translucent in the center and are golden brown, 3 to 4 minutes. Let the oil come back up to temperature between batches. Drain the nuggets on a paper towel-lined plate before serving.