For a long time it looked as though the Gunners, who included youngsters Ainsley Maitland-Niles, Reiss Nelson and Joe Willock from the start, would have to settle for a point in Serbia, even after left-back Milan Rodić was sent off for a second yellow late on.
But five with five minutes remaining the Gunners made their numerical superiority count. The impressive Jack Wilshere played a neat one-two with Theo Walcott and the ball fell invitingly for Giroud to lift a bicycle kick into the top corner.
Red Star almost get what they set out for
There was little to no subtlety about the way the hosts set up. Having seen Arsenal easily dispatch Cologne and BATE Borisov in their opening group games, the Serbians set out to frustrate and avoid defeat.
Without the ball they were organised and sat deep in two banks of four, asking Wenger's side to try to play through them. The Gunners, more than most Premier League sides, are used to that and more than capable of unpicking a stubborn defence.
But with a hotch-potch side and little pace or movement up front in Everton transfer target Giroud, the Gunners huffed and puffed but could not blow the hosts' house down. And he was not helped by the lack of deliveries into the penalty area.
That was until five minutes from the end.
The one bit of quality in the whole game saw the ball headed into Giroud's path by Walcott and the Gunners' No.12 guided a laboured bicycle kick home to settle the tie.
The turn and pass from Wilshere ?
The finish from Giroud ?
What a goal from Arsenal ? pic.twitter.com/jVYnzVmeMd
— Football on BT Sport (@btsportfootball) October 19, 2017
That moment aside it was the wrong game for Giroud. The Frenchman is many things, including a very handy substitute, but he does not have the pace of Walcott, Manchester City transfer target Alexis Sánchez or Alexandre Lacazette to get in behind and stretch the hosts.
Theo Walcott wasted at No.10
The England international has had a tough season of it. Yet to start a Premier League fixture, the Europa League has been Walcott's domain and chance ot impress.
But stuck behind Giroud he was caught in the traffic. Arsenal's build-up play was ponderous and predictable and that did not suit Walcott whose most saleable asset is his searing pace.
He did his best, drifting wide whenever possible as highlighted by the average position of his touches (No.14) below.
Really the Gunners could have done with inviting their hosts onto them a bit more and trying to hit them on the counter with Walcott's pace their not so secret weapon.
But nominally playing level with Wilshere the winger-cum-forward could not do what he does best.
11 – Theo Walcott’s assist was his 11th for Olivier Giroud in all comps; only Mesut Ozil has provided more assists for Giroud (15). Synergy. pic.twitter.com/eYk5WJlk5R
— OptaJoe (@OptaJoe) October 19, 2017
It was, despite that, Walcott who provided the assist for Giroud's winner, leaping to nod the ball across to the French international who lofted a bicycle kick into the top corner.
Rob Holds things up
Wenger's attitude towards the Europa League has been summed up in his team selections for the competition. It is clearly not one of his priorities so putting a left-footed player on the left of the back three was not going to be make-or-break, especially with a plethora of defensive injuries governing his selection policy.
But it was noticeable the difference having the right-footed Holding on the left of the defensive trio made. Nacho Monreal, a left footer, is first-choice in that position.
When the Spaniard receives the ball he does so with his left. It means he can open up his body, push the ball forward and drive the Gunners up the field. Or he can play out and find the left wing-back.
Holding, though, was reluctant to do that which limited the passing channels for the visitors and made them predictable at times, either knocking the ball into midfield or playing back across the defensive line. It's a small thing but these can be the margins.
Wilshere reinvented as a No.10
While Walcott struggled to impact the game from a more central role, Wilshere thrived. His average touch map shows he (No.10) did most of his work close to Giroud (12) and that was where the goal came from.
It was the England international's drive, awareness and intelligence which finally found a way through the resolute Red Star defence.
Darting between two defenders he played a one-two with Walcott who, in turn, nodded the ball across the six-yard box for Giroud to turn home a late winner.
In a past life Wilshere has played a lot deeper for Arsenal, frequently as one of the two in 4-2-3-1 in front of the back four. But further up the field he can do more damage and that proved to be the case in Serbia.