Hi Jamie,
Here at University of Concepcion we have the following approach. There is a course called Applied Ocean Modelling mostly for advanced undergraduates and the rare desperate graduate student whose advisor told her/him to add a chapter with a numerical model in their thesis. The objective is to teach the practical use of an ocean model, by default it has been ROMS_AGRIF (now CROCO), but the idea is the same for FVCOM, ROMS, or another model, basically to focus on the second part of what you are thinking, ROMS configuration and basic experiments. Those students that actually continue with an interest in ocean modelling (2 out of 10 here) might then (or before) take a course on finite differences, build a toy model, etc, and hopefully do a research topic of undergraduate thesis with the model. The point is to allow them to acquire experience from a young age. It is a bit of a black box approach.
The topics that are usually covered are when focusing on general knowledge/basic concepts and technical language are ("theory" class)
Introduction to numerical modelling (language, history)
Overview of basic tools in Matlab/Linux
Pre-Postprocessing tools
Grid Generation
Atmospheric Forcing and Ocean Reanalysis
Model Validation
Lagrangian Tools
Biogeochemical modules
In practical term I try to show them how to do the following cases with a known configuration (Benguela domain), and ask them to repeat this in a domain of their choice.
0) Install and compile the model in their laptop
1) Create a Climatological Simulation
2) Make all sort of descriptive plots from a model output
3) Make validation plots using model output and satellite SST.
4) Change vertical resolution, horizontal resolution, model time step, time step for model output.
5) Do a hindcast (BC from a reanalysis)
6) Do a forecast
7) Add tides, NPZD module, plot the new variables
Use a lagrangian model (OpenDrift)
in between we discuss a few papers so they learn how to identify BC, IC, spin up in a model description.
If they advance fast, a good idea is to ask them to repeat the same, but for a new domain. This usually creates new errors and they learn 1) how do they look like, 2) how to solve them.
Hope this helps