In many online learning paradigms, such as the Expert and Bandit problems, each single decision point (either expert or arm) is evaluated and learned in a homogeneous way. Motivated by the learning paradigms of some special features, my work introduces a heterogeneous learning structure, which groups similar decision points and then accordingly runs differentiated learning algorithms of different types/parameters. In some cases involving in massive similar or correlated decision points, we prove that this method can speed up the learning process. For example, with a recursive learning structure applied to the Exponential Weighting method, the proposed algorithm can touch the tight regret lower bound for the Lipschitz Expert problem. This is the first optimal algorithm having been reported in the literature.