This is a modification of a mV Source that can be whipped up easily. You could use a DPM or Multimeter to read the output. The ability of this circuit to perform well depends on the quality of all the MFR resistors and the MultiTurn Pot. Use a Bourns 10T Pot.
Good Soldered Joints, Keep all Resistors and temperature sensitive parts from Transformer and Regulators. Keep Ripple in power supplies low, no EMI tolerated. If you have problems, make a Battery Powered Unit. Shield well in case you are in a Electrically Noisy environment.
In this link see at bottom this circuit millivolt source, pdf.
I have put a better offset null, OP07 has around 75uV offset error which may show as +/- 1 count error on 4 1/2 DPM 19999 counts. You can skip it if you are using a 3 1/2 digit DPM as the error will not show, even it 4 1/2 it may be upto 2 counts only.
R9, P4 and R10 are for balance and offset as you said you can use it that way. (old circuit)
C7 can be a low leakage plastic cap, even a tantalum electrolytic is ok, aluminum electrolytic may cause a very small error.
Q1 can be any npn that can take 100mA current, do not use RF devices, 2N2222 is best.
If you use a DPM protect DPM inputs with clamping diodes or zeners or an error in bread-boarding may send +/- 12V to DPM and it may be damaged. Some DPMs come with protection like DMMs. use the circuit in del2003.pdf in analog section to make a 4 1/2 DPM.
Also in 2000mV range do not short outputs as the Q1 may get damaged, and in 200mV and 20mV range the output impedance is 10 ohms which is good for calibrating any high input impedance instrumentation like a process indicator etc. loading with 100K 10K will cause error. Most instruments are very high impedance so it is fine.