Проверьте отправляют пакет против полученных пакетов?
Доверие http://www.wisegeek.com/how-big-is-the-internet.htm, кажется, что Google индексировал 200 терабайт. Если это точно, со связью на 100 Мбит/с необходимо смочь получить 200 ТБ через год. Но у Вас будет индекс с некоторыми данными, которые полностью устарели... Сервер будет полностью зависеть Вашей производительности поискового робота, и способа, которым Вы храните данные! (Database использовал...),
Так или иначе большей проблемой будет устройство хранения данных, я думаю.
Наконец, никто здесь не сможет дать Вам корректный ответ без полного набора информации. Необходимо нанять специалиста в этом очень определенном домене.
I have machine A -> machine B (one-way connection from A to B) . I need to send massive TCP packets from A to B and validate that all packets revived. Do you have any recommendation for tools?
I recommend TCP, which ensures reliable, ordered delivery of data.
(You are asking us for a tool to do what the underlying protocol already does.)
Also note that TCP is inherently bidirectional: ACKs have to get sent back from Machine B to machine A. A "one-way connection" is impossible - No ACKs means transmission will stall forever.
I think on upload big file from A to B (do this when they has two-way connection) dump the traffic from A to B and than replay it with tcpreplay (i will create 100 threads that will replay it in order to have massive tcp stream). In order to validate i will analyze the tcpdump on the receiver to check if its has the same excepted amount of packets as sent.
TCP doesn't work that way. The packet count over the wire may be different due to differing window/MTU sizes, dropped-and-retransmitted packets, delayed ACKs (resulting in double transmission of a few packets), etc.
You can't just shove the same data over the wire and expect it to work exactly the same way -- In practice this often works (and is the basis of replay attacks), but TCP is governed by a state machine, and you really need to run through that for each connection.
If you need to generate a large amount of TCP traffic and don't care about the data you should simply start multiple streams sending the data to listeners on the remote system (which presumably can just discard it).
If you care about the data you should do as lasrks suggested and checksum the received data (in its entirety) after it's exited the TCP/IP stack...