We propose incorporation of adaptive transmission probability to ALOHA-Q, which is a framed slotted ALOHA-based random access protocol ingeniously integrating Q-learning for slot selection in a frame. The transmission probability is also adaptively controlled based on Q-learning. Performance of the proposed protocol is confirmed by means of computer simulation. Numerical results show that the proposed protocol can mitigate performance degradation of ALOHA-Q under overloaded traffic condition and exhibits comparable performance to ALOHA-Q for moderate traffic condition.