Honestly, you should be contacting a wildlife rescue in your area. These places exist for the purpose of taking in injured or abandoned wildlife that people find. A wild mouse is NOT a good pet; they can carry many diseases and parasites and, as this one is a baby, it is very fragile and should be in the hands of professionals who know what care to give it.
Here is a link to the PA Wildlife Rehabilitators website. It will help you find a rehab center in your area:
Pennsylvania Association of Wildlife Rehabilitators
Threaded Mode