Net worth includes the value of what you own minus what you owe. Your bank balance only shows part of that picture.
An example with numbers
If you receive $1,000 of extra income and use it to pay down principal, your bank balance returns to where it was, but you owe $1,000 less. If instead you pay down debt with $1,000 you had already saved, cash and debt fall by the same amount and net worth does not change.
Try it with your numbers
Draw up a balance sheet before and after both operations, using the same value for the rest of what you own.
Example figures for learning purposes. They are not a recommendation and do not replace professional advice.