Differences between on-chain and off-chain transactions

Ledger blockchains introduced a way to distribute and decentralize finances, creating an ecosystem without central authorities. Payments sent between wallets in the blockchain stay recorded forever in the public transactions book. And, what can you do if you'd like to transfer funds to a friend without having your transaction public forever?