We introduce a new nonlinear scalarizing functional in set optimization with respect to variable domination structures. By means of this functional, we characterize solutions of set optimization problems, where the solution concept is given by the set approach. We also investigate the relationship between the well-posedness property of a set-valued problem and the Tykhonov well-posedness property of the scalarized problem by means of the proposed scalarizing functional. Also, two classes of well-posed set optimization problems with respect to variable domination structures are identified. Finally, we apply our results to uncertain vector optimization problems.