Recently, the industrial demand for SWIR detector, essential for the development of 3D image sensor for next-generation automotive LiDAR system or VR/AR device application, is rapidly increasing. Here, we report that successful demonstration of SWIR organic photodiode based on the polaron absorption of doped polymer can be achieved by modifying the ratio between bound and free polaron via simple sequential doping process. We propose that bound-to-free polaron transition undergoes when SWIR light is illuminated, releasing additional holes as the photocurrent. From a well-known polymer donor P3HT and dopant F4TCNQ, the ratio of bound over free polaron are maximized by changing the dopant solution from acetonitrile to chlorobenzene. Consequently, the optimized photodiode yielded a high external quantum efficiency over 1250% and specific detectivity over 5x1011 Jones at 1800 nm, which was never reported in photodiode using organic materials as the active layer.