That's the rumor.

Transgender NYC nightlife fixture Amanda Lepore dished about a dalliance with a famous rapper in her new memoir Doll Parts, and based on the details she gave, everyone is speculating that it's Kanye West.

"Ms. Lepore describes having an assignation with a famous rapper whose songs often play in the clubs she frequents. Soon after, she writes, he got married, and ‘I couldn’t help but think that his wife had a similar body type to me,'" the New York Times wrote in their profile of the book. 'Ye did get married to Kim Kardashian, whose voluptuous physique is undeniably similar to the blonde bombshell.

A KW source shot this down really quickly, though. "That is not within the realm of possibility," the source told Page Six.

Neither Kanye or Kim have responded to the chatter directly, and we doubt they're going to!