Manager Blair Dreelan originally put the boyband together, before the addition of George Shelley, calling them Triple J - but now the band have parted ways with Dreelan in order to sign a deal with Louis Walsh instead.
A source told Celebritain, "Louis is so excited about Union J.
"He genuinely believes they can be as big as One Direction and The Wanted but he needs control over them - having another manager involved just wouldn't work."
It's thought that Union J will confirm the deal this weekend, after The X Factor final has taken place.
Louis has previously said that he thinks Union J are set to become the 'next big boyband', and that the boys will have a 'massive career'.
Union J with Louis Walsh and Dermot O'Leary, finding out they'd been eliminated.
Speaking to Yahoo OMG! at the X Factor press conference, Louis said, "I believe they have something special. I know they are going to have a massive career, they are going to be like JLS or One Direction."
He continued, "[They have] no worries in the world, I know they'll be back on the show next year as guests. They're the next big boy band.
"I think they're going to do well. There is something about them."
Watch Union J singing for survival below:
Below, the biggest reality TV music flops