Conditional random field methods (CRFs) have gained popularity for image labeling tasks in recent years. In this paper, we describe an alternative discriminative approach, by extending the large margin principle to incorporate spatial correlations among neighboring pixels. In particular, by explicitly enforcing the sub modular condition, graph-cuts is conveniently integrated as the inference engine to attain the optimal label assignment efficiently. Our approach allows learning a model with thousands of parameters, and is shown to be capable of readily incorporating higher-order scene context. Empirical studies on a variety of image datasets suggest that our approach performs competitively compared to the state-of-the-art scene labeling methods.