Ising machines can find optimum or quasi-optimum solutions of combinatorial optimization problems efficiently and effectively. It is known that, when a good initial solution is given to an Ising machine, we can finally obtain a solution closer to the optimal solution. However, several Ising machines cannot directly accept an initial solution due to its computational nature. In this paper, we propose a method to give quasi-initial solutions into Ising machines that cannot directly accept them. The proposed method gives the positive or negative external magnetic field coefficients (magnetic field controlling term) based on the initial solutions and obtains a solution by using an Ising machine. Then, the magnetic field controlling term is re-calculated every time an Ising machine repeats the annealing process, and hence the solution is repeatedly improved on the basis of the previously obtained solution. The proposed method is applied to the capacitated vehicle routing problem with an additional constraint (constrained CVRP) and the max-cut problem. Experimental results show that the total path distance is reduced by 5.78% on average compared to the initial solution in the constrained CVRP and the sum of cut-edge weight is increased by 1.25% on average in the max-cut problem.