The Belgium international replaced Costa after 58 minutes and he got behind Wolfsberger with ease as he offered a different threat to his team-mate.
Batshuayi immediately looked dangerous with his first touches, showing his pace as a ball was looped over the top by Ruben Loftus-Cheek but the shot was deflected wide, while he was also denied by the offside trap.
Goals on the night came from Bertrand Traore in the 41st minute, Loftus-Cheek in the 83rd minute and Nathaniel Chalobah with the very last kick of the game, but Batshuayi and Costa could have easily been the guys to apply the finishing touches.
Conte was willing his star strikers to score as he gestured wildly after Batshuayi's offside goal and made sure to congratulate Costa, who improved on his last game in Vienna, as he was substituted.
Chelsea will be able to find room for two strikers next season and Costa will remain his manager's first choice after being described as "one of the best in the world" by the Italian.
Conte would like to work more on winger and strikers , he announced the new formation to be 4-2-4.