Continuity, making calls made on ipad to iphoneand taking notes

[MacNchedder]

I like to use the iPad to make calls on my iphone using continuity and often then use the Apple pencil and the Notes app to take notes during the call. HOWEVER, the sound of the pencil on the iPad can be annoying to the other person on the call. Anyone know of a way to get the call over to the iPhone to prevent the annoying sounds? Apple support was not aware of any way. I even tried turning Continuity off on the iPad (Settings/Facetime/Calls from iPhone), but that disconnected the call. I will cross post this in the ipad forum also. Thanks!