Complicated answer, and some historical background.
A watch can be somewhat temperature sensitive. This has been greatly reduced by the use of Elinvar in balance springs, but it is not eliminated,
By way of history for others, as taken from Wikipedia: "Around 1765, Pierre Le Roy ...invented the compensation balance, which became the standard approach for temperature compensation in watches and chronometers. In this approach, the shape of the balance is altered, or adjusting weights are moved on the spokes or rim of the balance, by a temperature-sensitive mechanism. This changes the moment of inertia of the balance wheel, and the change is adjusted such that it compensates for the change in modulus of elasticity of the balance spring. The compensating balance design of Thomas Earnshaw, which consists simply of a balance wheel with bimetallic rim, became the standard solution for temperature compensation.
"While the compensating balance was effective as a way to compensate for the effect of temperature on the balance spring, it could not provide a complete solution. The basic design suffers from "middle temperature error": if the compensation is adjusted to be exact at extremes of temperature, then it will be slightly off at temperatures between those extremeVarious "auxiliary compensation" mechanisms were designed to avoid this, but they all suffer from being complex and hard to adjust.
"Around 1900, a fundamentally different solution was created by Charles Édouard Guillaume, inventor of elinvar. This is a nickel-steel alloy with the property that the modulus of elasticity is essentially unaffected by temperature. A watch fitted with an elinvar balance spring requires either no temperature compensation at all, or very little. This simplifies the mechanism, and it also means that middle temperature error is eliminated as well, or at a minimum is drastically reduced."
Still, I once saw a chart showing errors of + or - about 5 seconds a day with Elinvar, depending on temperature --especially if a watch "bakes" or is subjected to freezing temperatures. Keep in mind that a 5 seconds variance/day is still only a .01% error, and de minimus.
That said, we can't tell whether it's your oils, the oiling, the time since the last oiling, or the balance spring causing this "error", or some combination of these factors. It's very difficult to diagnose via Internet description, but also with respect it doesn't sound material to me.