In 2004, Howard was named one of Teen People's Sexiest Guys Ever after he became the NBA's No. 1 draft pick Frazer Harrison/Getty; Disney/Eric McCandles/Getty According to , sexy isn't defined by just your looks. While looking back at how far he's come since being named one of in 2004, the former NBA player — who is on season 33, partnering with — admitted that he's "never thought" about being admired in that way.

"I've heard people say, 'oh my God, you're my celebrity crush' and stuff like that," he tells PEOPLE, admitting that the idea is still "crazy" to think about. "It is an honor if somebody sees you and they like what they see. They say beauty is in the eye of the beholder.

" Related: While everyone's definition of sexy may vary, no matter what, Howard, 38, says "confidence" is "key." "You've got to be confident..

. If you don't have that confidence, how is somebody else going to have confidence in you?" he explains. "You got to believe in yourself.

And that belief, people can feel it. That's the energy that I'm talking about." The athlete continues, "Your aura stands out and I think when someone has that energy, when they walk into the room or when you're around people, people can sense it.

They can feel it and then they know, 'Hey, that's a sexy person.'" Nowadays, when he thinks about what kind of qualities make someone sexy, Howard says it's all about how people "carry themselves" and their "energy." Related: "Like I said, [it's about] their aura," he says.

"You .