Unfortunately the way the Testfor command works is by checking if a set rule you give is true and then sending a redstone signal (via comparator) so the command block would need a constant redstone pulse to refresh the command.
The moment the rule becomes untrue the command block will remove the redstone signal. So in your case the moment you move the item to another inventory slot
