temperature and the MC

I'm running the MC off  a 12V battery pack and using two voltage regulators to provide 9.6V for the MC and 4.8V for external servo power source.  All this is mounted in a high tech lunchbox.

The 9.6V regulator heats up a lot.  It's on a heat sink, but the temperature in the lunchbox is bound to rise.  Is this something I should worry about?

This is in a sealed box?  You might want to reconsider that anyway.  There are parts on the board (regulators, driver chips, etc.) which produce their own heat which if not dissipated would accumulate in there anyway without any ventilation or some other way to get the heat outside of the box.  In practice, the heat involved may be little enough to travel through the box to the outside (you could get lucky), but it's not a given, especially if the box is in a warm location or has the Sun shining onto it.

You might start having problems with die temperatures over 50C.  Die temperatures of parts that radiate heat are likely to be above that of the surrounding air or even that of a heat sink.

Thermal transfer is heavily dependent on the difference in temperature between the object that is trying to shed heat and the medium (air, for instance) into which it is trying to dump it.  That's why you want the air to be moving.  Otherwise, you're trying to dump heat into air that you've already dumped heat into, so the temperature difference will be less than you need it to be.  This is also true when using heat sinks.  Once the heat is in the sink, it still has to be transferred to the air.  Air cooling will never yield a temperature lower than that of the air, but ventilation and forced-air bring in new, not-yet-heated air and push out the heated air.

Adding more heat sources certainly doesn't help.

Put the 9.6V regulator's heatsink on the outside of the box if you can.  If the box is metal, you can use the side of it as a heat sink, but you will really need some way of moving outside air across it to carry the heat away.  The laminar air flow on the outside of the box which results from warm air rising will only accomplish so much.  You still have to keep the air inside the box as cool as you can.

There are applications which require a weather-sealed enclosure, and the measures that must be taken to prevent heat accumulation can be quite involved.  Heat pipes, heatsinks on the outside with cooling media being moved post them, etc.

If feasible, you may want to consider using switching regulators instead of linear ones.  They cost more, but waste a lot less of the energy as heat.  Linear regulators turn the voltage difference between the input and output directly into heat as a function of the current being drawn through the output.  Switchers emit heat as well, but for nontrivial loads, much less of it.

if you're worried abuot heat, just hook up a little computer fan inside your box that will push air in from the outside so you can get some circulation.

You could always run some tests as well to see what the air temperature does inside the box over time.