It is of great significance to design a stable and reliable routing protocol for Vehicular Ad Hoc Networks (VANETs) that adopt Vehicle to Vehicle (V2V) communications in the face of frequent network topology changes. In this paper, we propose a hybrid routing algorithm, RCRIQ, based on improved Q-learning. For an established cluster structure, the cluster head is used to select the gateway vehicle according to the gateway utility function to expand the communication range of the cluster further. During the link construction stage, an improved Q-learning algorithm is adopted. The corresponding neighbor vehicle is chosen according to the maximum Q value in the neighbor list. The heuristic algorithm selects the next-hop by the maximum heuristic function value when selecting the next-hop neighbor node. The above two strategies are comprehensively evaluated to determine the next hop. This way ensures the optimal selection of the next hop in terms of reachability and other communication parameters. Simulation experiments show that the algorithm proposed in this article has better performance in terms of routing stability, throughput, and communication delay in the urban traffic scene.

Machine-to-Machine (M2M) communication plays a pivotal role in the evolution of Internet of Things (IoT). Cellular networks are considered to be a key enabler for M2M communications, which are originally designed mainly for Human-to-Human (H2H) communications. The introduction of M2M users will cause a series of problems to traditional H2H users, i.e., interference between various traffic. Resource allocation is an effective solution to these problems. In this paper, we consider a shared resource block (RB) and power allocation in an H2H/M2M coexistence scenario, where M2M users are subdivided into delay-tolerant and delay-sensitive types. We first model the RB-power allocation problem as maximization of capacity under Quality-of-Service (QoS) constraints of different types of traffic. Then, a learning framework is introduced, wherein a complex agent is built from simpler subagents, which provides the basis for distributed deployment scheme. Further, we proposed distributed Q-learning based autonomous RB-power allocation algorithm (DQ-ARPA), which enables the machine type network gateways (MTCG) as agents to learn the wireless environment and choose the RB-power autonomously to maximize M2M pairs' capacity while ensuring the QoS requirements of critical services. Simulation results indicates that with an appropriate reward design, our proposed scheme succeeds in reducing the impact of delay-tolerant machine type users on critical services in terms of SINR thresholds and outage ratios.

Multi-hop V2V communication is a fundamental way to realize data transmission in Vehicular Ad-hoc Networks (VANET). It has excellent potential in intelligent transportation systems and automatic vehicle driving, and positively affects the safety, reliability, and comfort of vehicles. With advantages in speed and trajectory, distribution along the route, size, etc., the urban buses have become prospective relay nodes for urban VANETs. However, it is a considerable challenge to construct stable and reliable (meeting the requirements of bandwidth, delay, and bit error rate) multi-hop routing because of the complexity of the urban road and bus line network in the communication area, as well as many unevenly distributed buses on the road, etc. Given this above, this paper proposes a new hierarchical routing algorithm based on V2V geographic topology segmentation. Urban hierarchical routing is divided into two layers. The first layer of routing is called coarse routing, which is composed of areas; the second layer of routing is called internal routing (bus routing within the area). Q-learning is used to formulate the sequence of buses that transmit information within each area. Details are as follows: Firstly, based on a city map containing road network information, the entire city is divided into small grids by physical streets. Secondly, based on an analysis of the characteristics of the adjacent grid bus lines, the grids with the same routing attributes are integrated into the same area, reducing the algorithm's computational complexity during route discovery. Then, for the calculated area set, a coarse route composed of the selected area is established by filtering out a group of areas satisfying from the source node to the destination node. Finally, the bus sequence between anchor intersections is selected within the chosen area, and a complete multi-hop route from the source node to the destination node is finally constructed. Sufficient simulations show that the proposed routing algorithm has more stable performance in terms of packet transmission rate, average end-to-end delay, routing duration, and other indicators than similar algorithms.