In a small saucepan, combine the milk, caramel topping and butter. Cook and stir over medium-low heat until butter is melted; add cinnamon and ginger. Stir a small amount of mixture into eggs; return all to the pan, stirring constantly.
Place bread cubes in a large bowl; gently stir in milk mixture. Let stand for 10 minutes. Gently stir in pears; transfer to a greased 11-in. x 7-in. baking dish. Bake, uncovered, at 350° for 20 minutes.
In a small bowl, combine flour and brown sugar; cut in butter until crumbly. Stir in toffee bits.
Sprinkle flour mixture over dish. Bake 20-25 minutes longer or until a knife inserted near the center comes out clean and pears are tender. Cool for 10 minutes before serving. Refrigerate leftovers. Yield: 12 servings.