I am reading the technical "specification" for stealth addresses, available here. The core idea is described in the following paragraph:
Using Elliptic curve Diffie-Hellman (ECDH) we can generate a shared secret that the payee can use to recover their funds. Let the payee have keypair Q=dG. The payor generates nonce keypair P=eG and uses ECDH to arrive at shared secret c=H(eQ)=H(dP). This secret could be used to derive a ECC secret key, and from that a scriptPubKey, however that would allow both payor and payee the ability to spend the funds. So instead we use BIP32-style derivation to create Q'=(Q+c)G and associated scriptPubKey.
I understand everything except the last sentence. As far as I can tell, Q and c are known to the payee and payer so the keypair Q'=(Q+c)G is also known to both. How does Q' allow for only the payee to spend the funds? How does BIP32-style derivation work?