Here's why:
* Direct objects receive the action of the verb.
* The verb in this sentence is "drove".
* Mrs. Chang is the subject performing the action.
* Us is the group of people being driven.
Therefore, "us" is the direct object because it's the thing being acted upon by the verb "drove".