The 20-year-old attacking midfielder spent the latter half of last season on loan with the Terries and had a starring role as they won promotion to the top flight via the Championship play-offs.
Manager David Wagner had hoped to bring Brown back to the Kirklees Stadium for their debut Premier League campaign but the potential deal is now complicated.
, Huddersfield intend to take Chelsea playmaker Kasey Palmer on loan again this summer. And that is where the stumbling block lies.
Premier League rules state Huddersfield can't take another player on loan from the Blues – meaning any deal involving Brown would have to be a permanent one.
Reports suggest Huddersfield are reluctant to part with the £8million required to land Brown, who they rate highly, as they have already splashed out £10million to sign Aaron Mooy from Manchester City.
Southampton have also shown an interest in taking England Under-20 star Brown to St Mary's on loan and could well take advantage of Huddersfield's predicament.
Is Brown worth the fuss?
Brown has made just one first-team appearance for Chelsea after joining the club in 2013 from West Bromwich Albion. As with many a Blues youngsters he's spent much of his time at the club out on loan, with stints at Vitesse Arnhem, Rotherham United and, most recently, Huddersfield.
He's gone a little under the radar but the coaching staff at Stamford Bridge clearly rate him. In 2014, then Chelsea manager José Mourinho made a bold claim about Brown and two of his teammates:
“My conscience tells me that if, for example, (Lewis) Baker, Brown, and (Dominic) Solanke are not national team players in a few years, I should blame myself,” Mourinho said.
“They are part of a process the club started without me. In this moment, we have players who will be Chelsea players. And when they become Chelsea players, they will become England players, almost for sure.
“At 16 or 17 I don’t think it’s good for them to go somewhere else to play, like in the Championship. At 17 they have to train with us and they have to learn with us. They will learn a lot.”
Brown has represented England at Under-16, Under-17, Under-19 and Under-20 levels, and even captained the Under-19 side.
Brown, like countless other Chelsea youngsters, was loaned out in order to get first-team football. During the first half of last season he was plying his trade for Rotherham, who ended the season bottom of the Championship.
During his six-month stay at the New York Stadium, Brown scored three goals and grabbed a further five assists in 17 starts.
He played across the front line and even lead the attack on a few occasions. His impressive form resulted in a move to promotion hopefuls Huddersfield.
The increase in quality around him resulted in Brown stepping up a level, and there's no reason it wouldn't happen if he moved to Southampton.
His stint at Huddersfield saw him finish with five goals in 15 starts. he didn’t manage to register an assist for the Terriers but it wasn't through lack of trying on his part. He created a number of golden chances his teammates failed to take advantage of.
His spell in the Championship saw him average two dribbles and 1.4 key passes per 90 minutes.
For context, only Nathan Redmond (1.5) and West Ham United transfer target Dušan Tadić (2) averaged more key passes and only Sofiane Boufal (3) completed more dribbles for the Saints last term.
Brown appears to fit the profile of a Southampton attacker. He shares similarities with Boufal, Redmond, Tadić and the recently departed Jay Rodriguez, now at West Bromwich Albion.
He's quick, aggressive in his nature, direct, skilful and can use either foot. Combining those strengths makes him unpredictable and in a league dominated by ultra-defensive teams you need that ingenuity from time-to-time.
However, unlike many with his traits, Brown doesn't just rely on his pace. He's intelligent and his movement is that of a much more experienced player.
The 20-year-old picks up pockets of space and his reading of the game makes him a goal threat in the final third.
In the series of pictures above you see Brown's anticipation, his vision and why he's a goal threat. The ball ricochets his way in the first picture but because he's already on his toes he's first to it, despite three opposition players initially being ahead of him.
He plays a clever pass between full-back and centre-back, who are both caught on their heels due to the transition from attack to defence. The right-back failed to track Brown and he's left in space to pull the strings.
The Chelsea man then continues his run into the penalty area and slots the ball home.
In the above stills you get to witness Brown's intelligence, awareness and ability on the ball.
He initially drops off into space and readies himself for a pass. What you don't see in the picture is Brown always checking over his should prior to receiving the pass to build up a picture of what's behind him.
As the pass is played he angles his body to receive it on the turn, taking him away from the oncoming player which allows him to attack the space in behind the midfield.
He carries it forward, cuts in onto his favoured right foot and curls a superb effort home.
It's a worldie.
It would be good business on Southampton's part to snap Brown up, develop him and, maybe, a few years down the line sell him for big money.
The Saints have a history in developing Chelsea players – Manchester City transfer target Ryan Bertrand and Oriol Romeu the most recent. It could well be three's a charm.