I think that there are two issues here that you need to resolve:
1. Customer - you need to ensure that your customer realizes the value that you add and perhaps you need to determine this first. If all you do is get online find a seller and then offer their prices to your buyer then perhaps it is difficult for the customer to see why they should be paying you money for something that they could easily do themselves. If you are however comparing products, prices, and visting and vetting suppliers, then there is a clear value to the customer in paying you a percentage to handle things for them.
So you need to make sure that you are adding a clear value for the customer - what are they paying for?
2. Supplier - as Blinky points out you also need to establish yourself with the supplier so that even if the customer does try to make direct contact with them the supplier will refer the customer back to you. Once again you need to show that add value through your services. You need to show the factory that you can get the orders without wasting their time. To do this you need to spend time with them preferably on the ground getting to know them and letting them get to know you. You need to understand prices and raw material prices and other pressures that affect the factory so that you can place pressure on them in the right places. They will respect the fact that you know the product and this will help them maintain loyalty to you even in the face of direct offers from your customer.
So again, showing that you are adding value is vital to ensuring that the supplier and the customer continue to choose you as their service provider.