What does "what did the five fingers say to the face" mean?

The question is a set-up to a joke from the Chappelle's Show skit "Charlie Murphy's True Hollywood Stories." It is posed to Charlie Murphy by Dave Chappelle while playing the character Rick James. The punch line to the joke is "Slap" and is said while Rick James slaps Charlie Murphy.

"Charlie Murphy's True Hollywood Stories" is a series of fictitious sketches detailing how Charlie Murphy met Rick James in the 1980s. As they become better acquainted, Rick and Charlie begin physically assaulting one another. The attacks begin playfully, but increase in seriousness and intensity. Following a fight in a hotel room, Rick and Charlie make amends, and the show implies that they smoke marijuana together.