diff --git a/LICENSE b/LICENSE index 8ece883b5..7ec3c06d2 100644 --- a/LICENSE +++ b/LICENSE @@ -4,8 +4,8 @@ OpenNetVM is distributed under the following BSD LICENSE: Copyright(c) - 2015-2016 George Washington University - 2015-2016 University of California Riverside + 2015-2017 George Washington University + 2015-2017 University of California Riverside All rights reserved. Redistribution and use in source and binary forms, with or without diff --git a/Makefile b/Makefile index ca2ee5cde..c94ac0f72 100644 --- a/Makefile +++ b/Makefile @@ -4,9 +4,9 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside -# 2016 Hewlett Packard Enterprise Development LP +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside +# 2016-2017 Hewlett Packard Enterprise Development LP # 2010-2014 Intel Corporation. # All rights reserved. # diff --git a/examples/Makefile b/examples/Makefile index a97f776c8..1e73522a4 100644 --- a/examples/Makefile +++ b/examples/Makefile @@ -4,9 +4,9 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside -# 2016 Hewlett Packard Enterprise Development LP +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside +# 2016-2017 Hewlett Packard Enterprise Development LP # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/aes_decrypt/Makefile b/examples/aes_decrypt/Makefile index ea284c312..7ed74f3d9 100644 --- a/examples/aes_decrypt/Makefile +++ b/examples/aes_decrypt/Makefile @@ -4,9 +4,9 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside -# 2016 Hewlett Packard Enterprise Development LP +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside +# 2016-2017 Hewlett Packard Enterprise Development LP # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/aes_decrypt/aes.c b/examples/aes_decrypt/aes.c index a08f0bc89..61848c10f 100644 --- a/examples/aes_decrypt/aes.c +++ b/examples/aes_decrypt/aes.c @@ -5,9 +5,9 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside - * 2016 Hewlett Packard Enterprise Development LP + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/aes_decrypt/aes.h b/examples/aes_decrypt/aes.h index b51db554d..1121c1629 100644 --- a/examples/aes_decrypt/aes.h +++ b/examples/aes_decrypt/aes.h @@ -5,9 +5,9 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside - * 2016 Hewlett Packard Enterprise Development LP + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/aes_decrypt/aesdecrypt.c b/examples/aes_decrypt/aesdecrypt.c index 5c47862e4..b54770e64 100644 --- a/examples/aes_decrypt/aesdecrypt.c +++ b/examples/aes_decrypt/aesdecrypt.c @@ -5,9 +5,9 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside - * 2016 Hewlett Packard Enterprise Development LP + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/aes_encrypt/Makefile b/examples/aes_encrypt/Makefile index 5a8559a47..61beeb601 100644 --- a/examples/aes_encrypt/Makefile +++ b/examples/aes_encrypt/Makefile @@ -4,9 +4,9 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside -# 2016 Hewlett Packard Enterprise Development LP +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside +# 2016-2017 Hewlett Packard Enterprise Development LP # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/aes_encrypt/aes.c b/examples/aes_encrypt/aes.c index a08f0bc89..61848c10f 100644 --- a/examples/aes_encrypt/aes.c +++ b/examples/aes_encrypt/aes.c @@ -5,9 +5,9 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside - * 2016 Hewlett Packard Enterprise Development LP + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/aes_encrypt/aes.h b/examples/aes_encrypt/aes.h index f2eb0dcac..9a697a293 100644 --- a/examples/aes_encrypt/aes.h +++ b/examples/aes_encrypt/aes.h @@ -5,9 +5,9 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside - * 2016 Hewlett Packard Enterprise Development LP + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/aes_encrypt/aesencrypt.c b/examples/aes_encrypt/aesencrypt.c index 6a8e7f7db..c26451446 100644 --- a/examples/aes_encrypt/aesencrypt.c +++ b/examples/aes_encrypt/aesencrypt.c @@ -5,9 +5,9 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside - * 2016 Hewlett Packard Enterprise Development LP + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/basic_monitor/Makefile b/examples/basic_monitor/Makefile index d8155e6e1..467b84fc5 100644 --- a/examples/basic_monitor/Makefile +++ b/examples/basic_monitor/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/basic_monitor/monitor.c b/examples/basic_monitor/monitor.c index cbb9e1013..126eee87d 100644 --- a/examples/basic_monitor/monitor.c +++ b/examples/basic_monitor/monitor.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/bridge/Makefile b/examples/bridge/Makefile index 04291c4e3..1624def31 100644 --- a/examples/bridge/Makefile +++ b/examples/bridge/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/bridge/bridge.c b/examples/bridge/bridge.c index 3494d3cd8..e2c08dc85 100644 --- a/examples/bridge/bridge.c +++ b/examples/bridge/bridge.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/Makefile b/examples/flow_table/Makefile index 353bf90ed..2fe5c163f 100644 --- a/examples/flow_table/Makefile +++ b/examples/flow_table/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/flow_table.c b/examples/flow_table/flow_table.c index 30b96e5fd..36b18aa76 100644 --- a/examples/flow_table/flow_table.c +++ b/examples/flow_table/flow_table.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/flow_table.h b/examples/flow_table/flow_table.h index 07a9e09d7..fcfd3591d 100644 --- a/examples/flow_table/flow_table.h +++ b/examples/flow_table/flow_table.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/msgbuf.c b/examples/flow_table/msgbuf.c index 1c10e8179..5b02a5c00 100644 --- a/examples/flow_table/msgbuf.c +++ b/examples/flow_table/msgbuf.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/msgbuf.h b/examples/flow_table/msgbuf.h index 0ec5edc34..7a2a3936c 100644 --- a/examples/flow_table/msgbuf.h +++ b/examples/flow_table/msgbuf.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/sdn.c b/examples/flow_table/sdn.c index e5c6feb29..3bff86aee 100644 --- a/examples/flow_table/sdn.c +++ b/examples/flow_table/sdn.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/sdn.h b/examples/flow_table/sdn.h index 1adba43f4..e17bcabc0 100644 --- a/examples/flow_table/sdn.h +++ b/examples/flow_table/sdn.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/sdn_pkt_list.h b/examples/flow_table/sdn_pkt_list.h index f387f545f..b0974c46e 100644 --- a/examples/flow_table/sdn_pkt_list.h +++ b/examples/flow_table/sdn_pkt_list.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/setupconn.c b/examples/flow_table/setupconn.c index 15940fa9d..0ac39c5a4 100644 --- a/examples/flow_table/setupconn.c +++ b/examples/flow_table/setupconn.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/flow_table/setupconn.h b/examples/flow_table/setupconn.h index abb441814..56248fa92 100644 --- a/examples/flow_table/setupconn.h +++ b/examples/flow_table/setupconn.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/ndpi_stats/Makefile b/examples/ndpi_stats/Makefile index fd93e172d..73622b658 100644 --- a/examples/ndpi_stats/Makefile +++ b/examples/ndpi_stats/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/ndpi_stats/ndpi_stats.c b/examples/ndpi_stats/ndpi_stats.c index 0e37b17d9..d51b0265b 100644 --- a/examples/ndpi_stats/ndpi_stats.c +++ b/examples/ndpi_stats/ndpi_stats.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/simple_forward/Makefile b/examples/simple_forward/Makefile index facc8ca5f..bff4b2a83 100644 --- a/examples/simple_forward/Makefile +++ b/examples/simple_forward/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/simple_forward/forward.c b/examples/simple_forward/forward.c index bdacf6d45..d458aece0 100644 --- a/examples/simple_forward/forward.c +++ b/examples/simple_forward/forward.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/speed_tester/Makefile b/examples/speed_tester/Makefile index 4488bf959..a065fe844 100644 --- a/examples/speed_tester/Makefile +++ b/examples/speed_tester/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/speed_tester/speed_tester.c b/examples/speed_tester/speed_tester.c index 784fa3c9e..6ebf1cd3a 100644 --- a/examples/speed_tester/speed_tester.c +++ b/examples/speed_tester/speed_tester.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/examples/test_flow_dir/Makefile b/examples/test_flow_dir/Makefile index b4daebd80..1bf1bd550 100644 --- a/examples/test_flow_dir/Makefile +++ b/examples/test_flow_dir/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/examples/test_flow_dir/test_flow_dir.c b/examples/test_flow_dir/test_flow_dir.c index 82adbe511..9d33e251a 100644 --- a/examples/test_flow_dir/test_flow_dir.c +++ b/examples/test_flow_dir/test_flow_dir.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm/Makefile b/onvm/Makefile index 43ad974f4..eaaf3c316 100644 --- a/onvm/Makefile +++ b/onvm/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # 2010-2014 Intel Corporation. # All rights reserved. # diff --git a/onvm/lib/Makefile b/onvm/lib/Makefile index 97cc19e7d..e9828ea12 100644 --- a/onvm/lib/Makefile +++ b/onvm/lib/Makefile @@ -4,10 +4,10 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # 2010-2014 Intel Corporation. -# 2016 Hewlett Packard Enterprise Development LP +# 2016-2017 Hewlett Packard Enterprise Development LP # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/onvm/onvm_mgr/Makefile b/onvm/onvm_mgr/Makefile index 80f7d5ac4..be3db3e63 100644 --- a/onvm/onvm_mgr/Makefile +++ b/onvm/onvm_mgr/Makefile @@ -4,8 +4,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # 2010-2014 Intel Corporation. # All rights reserved. # diff --git a/onvm/onvm_mgr/main.c b/onvm/onvm_mgr/main.c index b6e627410..4513d9471 100644 --- a/onvm/onvm_mgr/main.c +++ b/onvm/onvm_mgr/main.c @@ -5,10 +5,10 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. - * 2016 Hewlett Packard Enterprise Development LP + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -144,18 +144,18 @@ static int rx_thread_main(void *arg) { uint16_t i, rx_count; struct rte_mbuf *pkts[PACKET_READ_SIZE]; - struct thread_info *rx = (struct thread_info*)arg; + struct queue_mgr *rx_mgr = (struct queue_mgr*)arg; RTE_LOG(INFO, APP, "Core %d: Running RX thread for RX queue %d\n", rte_lcore_id(), - rx->queue_id); + rx_mgr->id); for (; worker_keep_running;) { /* Read ports */ for (i = 0; i < ports->num_ports; i++) { - rx_count = rte_eth_rx_burst(ports->id[i], rx->queue_id, \ + rx_count = rte_eth_rx_burst(ports->id[i], rx_mgr->id, \ pkts, PACKET_READ_SIZE); ports->rx_stats.rx[ports->id[i]] += rx_count; @@ -165,7 +165,7 @@ rx_thread_main(void *arg) { if (!num_nfs) { onvm_pkt_drop_batch(pkts, rx_count); } else { - onvm_pkt_process_rx_batch(rx, pkts, rx_count); + onvm_pkt_process_rx_batch(rx_mgr, pkts, rx_count); } } } @@ -182,26 +182,26 @@ tx_thread_main(void *arg) { struct onvm_nf *nf; unsigned i, tx_count; struct rte_mbuf *pkts[PACKET_READ_SIZE]; - struct thread_info* tx = (struct thread_info*)arg; + struct queue_mgr *tx_mgr = (struct queue_mgr *)arg; - if (tx->first_nf == tx->last_nf - 1) { + if (tx_mgr->tx_thread_info->first_nf == tx_mgr->tx_thread_info->last_nf - 1) { RTE_LOG(INFO, APP, "Core %d: Running TX thread for NF %d\n", rte_lcore_id(), - tx->first_nf); - } else if (tx->first_nf < tx->last_nf) { + tx_mgr->tx_thread_info->first_nf); + } else if (tx_mgr->tx_thread_info->first_nf < tx_mgr->tx_thread_info->last_nf) { RTE_LOG(INFO, APP, "Core %d: Running TX thread for NFs %d to %d\n", rte_lcore_id(), - tx->first_nf, - tx->last_nf-1); + tx_mgr->tx_thread_info->first_nf, + tx_mgr->tx_thread_info->last_nf-1); } for (; worker_keep_running;) { /* Read packets from the NF's tx queue and process them as needed */ - for (i = tx->first_nf; i < tx->last_nf; i++) { + for (i = tx_mgr->tx_thread_info->first_nf; i < tx_mgr->tx_thread_info->last_nf; i++) { nf = &nfs[i]; if (!onvm_nf_is_valid(nf)) continue; @@ -211,15 +211,15 @@ tx_thread_main(void *arg) { /* Now process the Client packets read */ if (likely(tx_count > 0)) { - onvm_pkt_process_tx_batch(tx, pkts, tx_count, nf); + onvm_pkt_process_tx_batch(tx_mgr, pkts, tx_count, nf); } } /* Send a burst to every port */ - onvm_pkt_flush_all_ports(tx); + onvm_pkt_flush_all_ports(tx_mgr); /* Send a burst to every NF */ - onvm_pkt_flush_all_nfs(tx); + onvm_pkt_flush_all_nfs(tx_mgr); } RTE_LOG(INFO, APP, "Core %d: TX thread done\n", rte_lcore_id()); @@ -287,36 +287,39 @@ main(int argc, char *argv[]) { signal(SIGTERM, handle_signal); for (i = 0; i < tx_lcores; i++) { - struct thread_info *tx = calloc(1, sizeof(struct thread_info)); - tx->queue_id = i; - tx->port_tx_buf = calloc(RTE_MAX_ETHPORTS, sizeof(struct packet_buf)); - tx->nf_rx_buf = calloc(MAX_NFS, sizeof(struct packet_buf)); - tx->first_nf = RTE_MIN(i * nfs_per_tx + 1, (unsigned)MAX_NFS); - tx->last_nf = RTE_MIN((i+1) * nfs_per_tx + 1, (unsigned)MAX_NFS); + struct queue_mgr *tx_mgr = calloc(1, sizeof(struct queue_mgr)); + tx_mgr->mgr_type_t = MGR; + tx_mgr->id = i; + tx_mgr->tx_thread_info = calloc(1, sizeof(struct tx_thread_info)); + tx_mgr->tx_thread_info->port_tx_bufs = calloc(RTE_MAX_ETHPORTS, sizeof(struct packet_buf)); + tx_mgr->nf_rx_bufs = calloc(MAX_NFS, sizeof(struct packet_buf)); + tx_mgr->tx_thread_info->first_nf = RTE_MIN(i * nfs_per_tx + 1, (unsigned)MAX_NFS); + tx_mgr->tx_thread_info->last_nf = RTE_MIN((i+1) * nfs_per_tx + 1, (unsigned)MAX_NFS); cur_lcore = rte_get_next_lcore(cur_lcore, 1, 1); - if (rte_eal_remote_launch(tx_thread_main, (void*)tx, cur_lcore) == -EBUSY) { + if (rte_eal_remote_launch(tx_thread_main, (void*)tx_mgr, cur_lcore) == -EBUSY) { RTE_LOG(ERR, APP, "Core %d is already busy, can't use for nf %d TX\n", cur_lcore, - tx->first_nf); + tx_mgr->tx_thread_info->first_nf); return -1; } } /* Launch RX thread main function for each RX queue on cores */ for (i = 0; i < rx_lcores; i++) { - struct thread_info *rx = calloc(1, sizeof(struct thread_info)); - rx->queue_id = i; - rx->port_tx_buf = NULL; - rx->nf_rx_buf = calloc(MAX_NFS, sizeof(struct packet_buf)); + struct queue_mgr *rx_mgr = calloc(1, sizeof(struct queue_mgr)); + rx_mgr->mgr_type_t = MGR; + rx_mgr->id = i; + rx_mgr->tx_thread_info = NULL; + rx_mgr->nf_rx_bufs = calloc(MAX_NFS, sizeof(struct packet_buf)); cur_lcore = rte_get_next_lcore(cur_lcore, 1, 1); - if (rte_eal_remote_launch(rx_thread_main, (void *)rx, cur_lcore) == -EBUSY) { + if (rte_eal_remote_launch(rx_thread_main, (void *)rx_mgr, cur_lcore) == -EBUSY) { RTE_LOG(ERR, APP, "Core %d is already busy, can't use for RX queue id %d\n", cur_lcore, - rx->queue_id); + rx_mgr->id); return -1; } } diff --git a/onvm/onvm_mgr/onvm_args.c b/onvm/onvm_mgr/onvm_args.c index cbe5240f0..a425f709a 100644 --- a/onvm/onvm_mgr/onvm_args.c +++ b/onvm/onvm_mgr/onvm_args.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * diff --git a/onvm/onvm_mgr/onvm_args.h b/onvm/onvm_mgr/onvm_args.h index aeee40059..12db94009 100644 --- a/onvm/onvm_mgr/onvm_args.h +++ b/onvm/onvm_mgr/onvm_args.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * diff --git a/onvm/onvm_mgr/onvm_init.c b/onvm/onvm_mgr/onvm_init.c index d74531bbc..d2ef0a9c1 100644 --- a/onvm/onvm_mgr/onvm_init.c +++ b/onvm/onvm_mgr/onvm_init.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * @@ -152,6 +152,8 @@ init(int argc, char *argv[]) { const struct rte_memzone *mz_nf; const struct rte_memzone *mz_port; const struct rte_memzone *mz_scp; + const struct rte_memzone *mz_services; + const struct rte_memzone *mz_nf_per_service; uint8_t i, total_ports, port_id; /* init EAL, parsing EAL args */ @@ -182,6 +184,21 @@ init(int argc, char *argv[]) { if (mz_port == NULL) rte_exit(EXIT_FAILURE, "Cannot reserve memory zone for port information\n"); ports = mz_port->addr; + + /* set up array for NF tx data */ + mz_services = rte_memzone_reserve(MZ_SERVICES_INFO, sizeof(uint16_t) * num_services, rte_socket_id(), NO_FLAGS); + if (mz_services == NULL) + rte_exit(EXIT_FAILURE, "Cannot reserve memory zone for services information\n"); + services = mz_services->addr; + for (i = 0; i < num_services; i++) { + services[i] = rte_calloc("one service NFs", + MAX_NFS_PER_SERVICE, sizeof(uint16_t), 0); + } + mz_nf_per_service = rte_memzone_reserve(MZ_NF_PER_SERVICE_INFO, sizeof(uint16_t) * num_services, rte_socket_id(), NO_FLAGS); + if (mz_nf_per_service == NULL) { + rte_exit(EXIT_FAILURE, "Cannot reserve memory zone for NF per service information.\n"); + } + nf_per_service_count = mz_nf_per_service->addr; /* parse additional, application arguments */ retval = parse_app_args(total_ports, argc, argv); @@ -372,18 +389,7 @@ init_shm_rings(void) { // use calloc since we allocate for all possible NFs // ensure that all fields are init to 0 to avoid reading garbage // TODO plopreiato, move to creation when a NF starts - services = rte_calloc("service to nf map", - num_services, sizeof(uint16_t*), 0); - for (i = 0; i < num_services; i++) { - services[i] = rte_calloc("one service NFs", - MAX_NFS_PER_SERVICE, sizeof(uint16_t), 0); - } - nf_per_service_count = rte_calloc("count of NFs active per service", - num_services, sizeof(uint16_t), 0); - if (services == NULL || nf_per_service_count == NULL) - rte_exit(EXIT_FAILURE, "Cannot allocate memory for service to NF mapping\n"); - - for (i = 0; i < MAX_NFS; i++) { + for (i = 0; i < MAX_NFS; i++) { /* Create an RX queue for each NF */ socket_id = rte_socket_id(); rq_name = get_rx_queue_name(i); diff --git a/onvm/onvm_mgr/onvm_init.h b/onvm/onvm_mgr/onvm_init.h index 71eddba92..a4317593d 100644 --- a/onvm/onvm_mgr/onvm_init.h +++ b/onvm/onvm_mgr/onvm_init.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * diff --git a/onvm/onvm_mgr/onvm_mgr.h b/onvm/onvm_mgr/onvm_mgr.h index e9daee3b0..fb6d8c78c 100644 --- a/onvm/onvm_mgr/onvm_mgr.h +++ b/onvm/onvm_mgr/onvm_mgr.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * @@ -80,13 +80,12 @@ #include "onvm_sc_mgr.h" #include "onvm_flow_table.h" #include "onvm_flow_dir.h" +#include "onvm_pkt_common.h" /***********************************Macros************************************/ -#define PACKET_READ_SIZE ((uint16_t)32) - #define TO_PORT 0 #define TO_NF 1 @@ -97,36 +96,4 @@ /* ID to be assigned to the next NF that starts */ extern uint16_t next_instance_id; -/* Client information, rx/tx/msg rings, nf_info, inst_id and stats */ -extern struct onvm_nf *nfs; - - -/*******************************Data Structures*******************************/ - - -/* - * Local buffers to put packets in, used to send packets in bursts to the - * NFs or to the NIC - */ -struct packet_buf { - struct rte_mbuf *buffer[PACKET_READ_SIZE]; - uint16_t count; -}; - - -/** Thread state. This specifies which NFs the thread will handle and - * includes the packet buffers used by the thread for NFs and ports. - */ -struct thread_info { - unsigned queue_id; - unsigned first_nf; - unsigned last_nf; - /* FIXME: This is confusing since it is non-inclusive. It would be - * better to have this take the first NF and the number - * of consecutive NFs after it to handle. - */ - struct packet_buf *nf_rx_buf; - struct packet_buf *port_tx_buf; -}; - #endif // _ONVM_MGR_H_ diff --git a/onvm/onvm_mgr/onvm_nf.c b/onvm/onvm_mgr/onvm_nf.c index 334e4f330..2dd43860a 100644 --- a/onvm/onvm_mgr/onvm_nf.c +++ b/onvm/onvm_mgr/onvm_nf.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * @@ -94,12 +94,6 @@ onvm_nf_stop(struct onvm_nf_info *nf_info); /********************************Interfaces***********************************/ -inline int -onvm_nf_is_valid(struct onvm_nf *nf) { - return nf && nf->info && nf->info->status == NF_RUNNING; -} - - uint16_t onvm_nf_next_instance_id(void) { struct onvm_nf *nf; @@ -170,23 +164,6 @@ onvm_nf_send_msg(uint16_t dest, uint8_t msg_type, void *msg_data) { return rte_ring_sp_enqueue(nfs[dest].msg_q, (void*)msg); } - -inline uint16_t -onvm_nf_service_to_nf_map(uint16_t service_id, struct rte_mbuf *pkt) { - uint16_t num_nfs_available = nf_per_service_count[service_id]; - - if (num_nfs_available == 0) - return 0; - - if (pkt == NULL) - return 0; - - uint16_t instance_index = pkt->hash.rss % num_nfs_available; - uint16_t instance_id = services[service_id][instance_index]; - return instance_id; -} - - /******************************Internal functions*****************************/ diff --git a/onvm/onvm_mgr/onvm_nf.h b/onvm/onvm_mgr/onvm_nf.h index 460ea02f1..e544ddc1d 100644 --- a/onvm/onvm_mgr/onvm_nf.h +++ b/onvm/onvm_mgr/onvm_nf.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * @@ -58,17 +58,6 @@ extern uint16_t next_instance_id; /********************************Interfaces***********************************/ -/* - * Interface checking if a given nf is "valid", meaning if it's running. - * - * Input : a pointer to the nf - * Output : a boolean answer - * - */ -int -onvm_nf_is_valid(struct onvm_nf *nf); - - /* * Interface giving the smallest unsigned integer unused for a NF instance. * @@ -100,16 +89,4 @@ int onvm_nf_send_msg(uint16_t dest, uint8_t msg_type, void *msg_data); -/* - * Interface giving a NF for a specific server id, depending on the flow. - * - * Inputs : the service id - a pointer to the packet whose flow help steer it. - * Output : a NF instance id - * - */ -uint16_t -onvm_nf_service_to_nf_map(uint16_t service_id, struct rte_mbuf *pkt); - - #endif // _ONVM_NF_H_ diff --git a/onvm/onvm_mgr/onvm_pkt.c b/onvm/onvm_mgr/onvm_pkt.c index aa6bec78a..108a8caea 100644 --- a/onvm/onvm_mgr/onvm_pkt.c +++ b/onvm/onvm_mgr/onvm_pkt.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * @@ -52,91 +52,18 @@ #include "onvm_pkt.h" #include "onvm_nf.h" - - -/**********************Internal Functions Prototypes**************************/ - - -/* - * Function to send packets to one port after processing them. - * - * Input : a pointer to the tx queue - * - */ -static void -onvm_pkt_flush_port_queue(struct thread_info *tx, uint16_t port); - - -/* - * Function to send packets to one NF after processing them. - * - * Input : a pointer to the tx queue - * - */ -static void -onvm_pkt_flush_nf_queue(struct thread_info *thread, uint16_t nf_id); - - -/* - * Function to enqueue a packet on one port's queue. - * - * Inputs : a pointer to the tx queue responsible - * the number of the port - * a pointer to the packet - * - */ -inline static void -onvm_pkt_enqueue_port(struct thread_info *tx, uint16_t port, struct rte_mbuf *buf); - - -/* - * Function to enqueue a packet on one NF's queue. - * - * Inputs : a pointer to the tx queue responsible - * the number of the port - * a pointer to the packet - * - */ -inline static void -onvm_pkt_enqueue_nf(struct thread_info *thread, uint16_t dst_service_id, struct rte_mbuf *pkt); - - -/* - * Function to process a single packet. - * - * Inputs : a pointer to the tx queue responsible - * a pointer to the packet - * a pointer to the NF involved - * - */ -inline static void -onvm_pkt_process_next_action(struct thread_info *tx, struct rte_mbuf *pkt, struct onvm_nf *nf); - - -/* - * Helper function to drop a packet. - * - * Input : a pointer to the packet - * - * Ouput : an error code - * - */ -static int -onvm_pkt_drop(struct rte_mbuf *pkt); - - /**********************************Interfaces*********************************/ void -onvm_pkt_process_rx_batch(struct thread_info *rx, struct rte_mbuf *pkts[], uint16_t rx_count) { +onvm_pkt_process_rx_batch(struct queue_mgr *rx_mgr, struct rte_mbuf *pkts[], uint16_t rx_count) { uint16_t i; struct onvm_pkt_meta *meta; struct onvm_flow_entry *flow_entry; struct onvm_service_chain *sc; int ret; - if (rx == NULL || pkts == NULL) + if (rx_mgr == NULL || pkts == NULL) return; for (i = 0; i < rx_count; i++) { @@ -159,71 +86,25 @@ onvm_pkt_process_rx_batch(struct thread_info *rx, struct rte_mbuf *pkts[], uint1 */ (meta->chain_index)++; - onvm_pkt_enqueue_nf(rx, meta->destination, pkts[i]); + onvm_pkt_enqueue_nf(rx_mgr, meta->destination, pkts[i]); } - onvm_pkt_flush_all_nfs(rx); + onvm_pkt_flush_all_nfs(rx_mgr); } void -onvm_pkt_process_tx_batch(struct thread_info *tx, struct rte_mbuf *pkts[], uint16_t tx_count, struct onvm_nf *nf) { - uint16_t i; - struct onvm_pkt_meta *meta; +onvm_pkt_flush_all_ports(struct queue_mgr *tx_mgr) { + uint16_t i; - if (tx == NULL || pkts == NULL || nf == NULL) - return; - - for (i = 0; i < tx_count; i++) { - meta = (struct onvm_pkt_meta*) &(((struct rte_mbuf*)pkts[i])->udata64); - meta->src = nf->instance_id; - if (meta->action == ONVM_NF_ACTION_DROP) { - // if the packet is drop, then is 0 - // and ! is 1. - nf->stats.act_drop += !onvm_pkt_drop(pkts[i]); - } else if (meta->action == ONVM_NF_ACTION_NEXT) { - /* TODO: Here we drop the packet : there will be a flow table - in the future to know what to do with the packet next */ - nf->stats.act_next++; - onvm_pkt_process_next_action(tx, pkts[i], nf); - } else if (meta->action == ONVM_NF_ACTION_TONF) { - nf->stats.act_tonf++; - onvm_pkt_enqueue_nf(tx, meta->destination, pkts[i]); - } else if (meta->action == ONVM_NF_ACTION_OUT) { - nf->stats.act_out++; - onvm_pkt_enqueue_port(tx, meta->destination, pkts[i]); - } else { - printf("ERROR invalid action : this shouldn't happen.\n"); - onvm_pkt_drop(pkts[i]); - return; - } - } -} - - -void -onvm_pkt_flush_all_ports(struct thread_info *tx) { - uint16_t i; - - if (tx == NULL) + if (tx_mgr == NULL) return; for (i = 0; i < ports->num_ports; i++) - onvm_pkt_flush_port_queue(tx, ports->id[i]); + onvm_pkt_flush_port_queue(tx_mgr, ports->id[i]); } -void -onvm_pkt_flush_all_nfs(struct thread_info *tx) { - uint16_t i; - - if (tx == NULL) - return; - - for (i = 0; i < MAX_NFS; i++) - onvm_pkt_flush_nf_queue(tx, i); -} - void onvm_pkt_drop_batch(struct rte_mbuf **pkts, uint16_t size) { uint16_t i; @@ -235,162 +116,3 @@ onvm_pkt_drop_batch(struct rte_mbuf **pkts, uint16_t size) { rte_pktmbuf_free(pkts[i]); } - -/****************************Internal functions*******************************/ - - -static void -onvm_pkt_flush_port_queue(struct thread_info *tx, uint16_t port) { - uint16_t i, sent; - volatile struct tx_stats *tx_stats; - - if (tx == NULL) - return; - - if (tx->port_tx_buf[port].count == 0) - return; - - tx_stats = &(ports->tx_stats); - sent = rte_eth_tx_burst(port, - tx->queue_id, - tx->port_tx_buf[port].buffer, - tx->port_tx_buf[port].count); - if (unlikely(sent < tx->port_tx_buf[port].count)) { - for (i = sent; i < tx->port_tx_buf[port].count; i++) { - onvm_pkt_drop(tx->port_tx_buf[port].buffer[i]); - } - tx_stats->tx_drop[port] += (tx->port_tx_buf[port].count - sent); - } - tx_stats->tx[port] += sent; - - tx->port_tx_buf[port].count = 0; -} - - -static void -onvm_pkt_flush_nf_queue(struct thread_info *thread, uint16_t nf_id) { - uint16_t i; - struct onvm_nf *nf; - - if (thread == NULL) - return; - - if (thread->nf_rx_buf[nf_id].count == 0) - return; - - nf = &nfs[nf_id]; - - // Ensure destination NF is running and ready to receive packets - if (!onvm_nf_is_valid(nf)) - return; - - if (rte_ring_enqueue_bulk(nf->rx_q, (void **)thread->nf_rx_buf[nf_id].buffer, - thread->nf_rx_buf[nf_id].count) != 0) { - for (i = 0; i < thread->nf_rx_buf[nf_id].count; i++) { - onvm_pkt_drop(thread->nf_rx_buf[nf_id].buffer[i]); - } - nf->stats.rx_drop += thread->nf_rx_buf[nf_id].count; - } else { - nf->stats.rx += thread->nf_rx_buf[nf_id].count; - } - thread->nf_rx_buf[nf_id].count = 0; -} - - -inline static void -onvm_pkt_enqueue_port(struct thread_info *tx, uint16_t port, struct rte_mbuf *buf) { - - if (tx == NULL || buf == NULL) - return; - - - tx->port_tx_buf[port].buffer[tx->port_tx_buf[port].count++] = buf; - if (tx->port_tx_buf[port].count == PACKET_READ_SIZE) { - onvm_pkt_flush_port_queue(tx, port); - } -} - - -inline static void -onvm_pkt_enqueue_nf(struct thread_info *thread, uint16_t dst_service_id, struct rte_mbuf *pkt) { - struct onvm_nf *nf; - uint16_t dst_instance_id; - - - if (thread == NULL || pkt == NULL) - return; - - // map service to instance and check one exists - dst_instance_id = onvm_nf_service_to_nf_map(dst_service_id, pkt); - if (dst_instance_id == 0) { - onvm_pkt_drop(pkt); - return; - } - - // Ensure destination NF is running and ready to receive packets - nf = &nfs[dst_instance_id]; - if (!onvm_nf_is_valid(nf)) { - onvm_pkt_drop(pkt); - return; - } - - thread->nf_rx_buf[dst_instance_id].buffer[thread->nf_rx_buf[dst_instance_id].count++] = pkt; - if (thread->nf_rx_buf[dst_instance_id].count == PACKET_READ_SIZE) { - onvm_pkt_flush_nf_queue(thread, dst_instance_id); - } -} - - -inline static void -onvm_pkt_process_next_action(struct thread_info *tx, struct rte_mbuf *pkt, struct onvm_nf *nf) { - - if (tx == NULL || pkt == NULL || nf == NULL) - return; - - struct onvm_flow_entry *flow_entry; - struct onvm_service_chain *sc; - struct onvm_pkt_meta *meta = onvm_get_pkt_meta(pkt); - int ret; - - ret = onvm_flow_dir_get_pkt(pkt, &flow_entry); - if (ret >= 0) { - sc = flow_entry->sc; - meta->action = onvm_sc_next_action(sc, pkt); - meta->destination = onvm_sc_next_destination(sc, pkt); - } else { - meta->action = onvm_sc_next_action(default_chain, pkt); - meta->destination = onvm_sc_next_destination(default_chain, pkt); - } - - switch (meta->action) { - case ONVM_NF_ACTION_DROP: - // if the packet is drop, then is 0 - // and ! is 1. - nf->stats.act_drop += !onvm_pkt_drop(pkt); - break; - case ONVM_NF_ACTION_TONF: - nf->stats.act_tonf++; - onvm_pkt_enqueue_nf(tx, meta->destination, pkt); - break; - case ONVM_NF_ACTION_OUT: - nf->stats.act_out++; - onvm_pkt_enqueue_port(tx, meta->destination, pkt); - break; - default: - break; - } - (meta->chain_index)++; -} - - -/*******************************Helper function*******************************/ - - -static int -onvm_pkt_drop(struct rte_mbuf *pkt) { - rte_pktmbuf_free(pkt); - if (pkt != NULL) { - return 1; - } - return 0; -} diff --git a/onvm/onvm_mgr/onvm_pkt.h b/onvm/onvm_mgr/onvm_pkt.h index 72eee9682..1f20cdab2 100644 --- a/onvm/onvm_mgr/onvm_pkt.h +++ b/onvm/onvm_mgr/onvm_pkt.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * @@ -66,21 +66,7 @@ * */ void -onvm_pkt_process_rx_batch(struct thread_info *rx, struct rte_mbuf *pkts[], uint16_t rx_count); - - -/* - * Interface to process packets in a given TX queue. - * - * Inputs : a pointer to the tx queue - * an array of packets - * the size of the array - * a pointer to the NF possessing the TX queue. - * - */ -void -onvm_pkt_process_tx_batch(struct thread_info *tx, struct rte_mbuf *pkts[], uint16_t tx_count, struct onvm_nf *nf); - +onvm_pkt_process_rx_batch(struct queue_mgr *rx_mgr, struct rte_mbuf *pkts[], uint16_t rx_count); /* * Interface to send packets to all ports after processing them. @@ -89,17 +75,7 @@ onvm_pkt_process_tx_batch(struct thread_info *tx, struct rte_mbuf *pkts[], uint1 * */ void -onvm_pkt_flush_all_ports(struct thread_info *tx); - - -/* - * Interface to send packets to all NFs after processing them. - * - * Input : a pointer to the tx queue - * - */ -void -onvm_pkt_flush_all_nfs(struct thread_info *tx); +onvm_pkt_flush_all_ports(struct queue_mgr *tx_mgr); /* diff --git a/onvm/onvm_mgr/onvm_stats.c b/onvm/onvm_mgr/onvm_stats.c index 84da528ec..3cd7d6729 100644 --- a/onvm/onvm_mgr/onvm_stats.c +++ b/onvm/onvm_mgr/onvm_stats.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * @@ -268,6 +268,7 @@ onvm_stats_display_nfs(unsigned difftime) { for (i = 0; i < MAX_NFS; i++) { if (!onvm_nf_is_valid(&nfs[i])) continue; + const uint64_t rx = nfs[i].stats.rx; const uint64_t rx_drop = nfs[i].stats.rx_drop; const uint64_t tx = nfs[i].stats.tx; diff --git a/onvm/onvm_mgr/onvm_stats.h b/onvm/onvm_mgr/onvm_stats.h index 575f01a71..9d1d46a4a 100644 --- a/onvm/onvm_mgr/onvm_stats.h +++ b/onvm/onvm_mgr/onvm_stats.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * diff --git a/onvm/onvm_nflib/Makefile b/onvm/onvm_nflib/Makefile index a65e10a66..3c76e3486 100644 --- a/onvm/onvm_nflib/Makefile +++ b/onvm/onvm_nflib/Makefile @@ -4,10 +4,10 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # 2010-2014 Intel Corporation. -# 2016 Hewlett Packard Enterprise Development LP +# 2016-2017 Hewlett Packard Enterprise Development LP # All rights reserved. # # Redistribution and use in source and binary forms, with or without @@ -49,7 +49,7 @@ RTE_TARGET ?= x86_64-native-linuxapp-gcc LIB = libonvm.a # all source are stored in SRCS-y -SRCS-y := onvm_pkt_helper.c onvm_sc_common.c onvm_sc_mgr.c onvm_flow_table.c onvm_flow_dir.c onvm_nflib.c +SRCS-y := onvm_pkt_helper.c onvm_sc_common.c onvm_sc_mgr.c onvm_flow_table.c onvm_flow_dir.c onvm_nflib.c onvm_pkt_common.c CFLAGS += $(WERROR_FLAGS) -O3 $(USER_FLAGS) diff --git a/onvm/onvm_nflib/onvm_common.h b/onvm/onvm_nflib/onvm_common.h index d16137516..2232dcfc9 100644 --- a/onvm/onvm_nflib/onvm_common.h +++ b/onvm/onvm_nflib/onvm_common.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation * All rights reserved. * @@ -54,6 +54,8 @@ #define MAX_SERVICES 16 // total number of unique services allowed #define MAX_NFS_PER_SERVICE 8 // max number of NFs per service. +#define PACKET_READ_SIZE ((uint16_t)32) + #define ONVM_NF_ACTION_DROP 0 // drop packet #define ONVM_NF_ACTION_NEXT 1 // to whatever the next action is configured by the SDN controller in the flow table #define ONVM_NF_ACTION_TONF 2 // send to the NF specified in the argument field (assume it is on the same host) @@ -94,6 +96,42 @@ static inline uint8_t onvm_get_pkt_chain_index(struct rte_mbuf* pkt) { * themselves are written by the NFs, so we have a distinct set, on different * cache lines for each NF to use. */ +/*******************************Data Structures*******************************/ + +/* + * Packets may be transported by a tx thread or by an NF. + * This data structure encapsulates data specific to + * tx threads. + */ +struct tx_thread_info { + unsigned first_nf; + unsigned last_nf; + struct packet_buf *port_tx_bufs; +}; + +/* + * Local buffers to put packets in, used to send packets in bursts to the + * NFs or to the NIC + */ +struct packet_buf { + struct rte_mbuf *buffer[PACKET_READ_SIZE]; + uint16_t count; +}; + +/* + * Generic data struct that tx threads and nfs both use. + * Allows pkt functions to be shared + * */ +struct queue_mgr { + unsigned id; + enum {NF, MGR} mgr_type_t; + union { + struct tx_thread_info *tx_thread_info; + struct packet_buf *to_tx_buf; + }; + struct packet_buf *nf_rx_bufs; +}; + struct rx_stats{ uint64_t rx[RTE_MAX_ETHPORTS]; }; @@ -112,8 +150,8 @@ struct port_info { }; /* - * Define a NF structure with all needed info, including - * stats from the NFs. + * Define a nf structure with all needed info, including + * stats from the nfs. */ struct onvm_nf { struct rte_ring *rx_q; @@ -176,6 +214,8 @@ struct onvm_service_chain { #define PKTMBUF_POOL_NAME "MProc_pktmbuf_pool" #define MZ_PORT_INFO "MProc_port_info" #define MZ_NF_INFO "MProc_nf_info" +#define MZ_SERVICES_INFO "MProc_services_info" +#define MZ_NF_PER_SERVICE_INFO "MProc_nf_per_service_info" #define MZ_SCP_INFO "MProc_scp_info" #define MZ_FTP_INFO "MProc_ftp_info" @@ -194,6 +234,8 @@ struct onvm_service_chain { #define NF_NO_IDS 6 // There are no available IDs for this NF #define NF_NO_ID -1 +#define ONVM_NF_HANDLE_TX 1 // should be true if NFs primarily pass packets to each other + /* * Given the rx queue name template above, get the queue name @@ -232,7 +274,14 @@ get_msg_queue_name(unsigned id) { snprintf(buffer, sizeof(buffer) - 1, _NF_MSG_QUEUE_NAME, id); return buffer; +} +/* + * Interface checking if a given NF is "valid", meaning if it's running. + */ +static inline int +onvm_nf_is_valid(struct onvm_nf *nf) { + return nf && nf->info && nf->info->status == NF_RUNNING; } #define RTE_LOGTYPE_APP RTE_LOGTYPE_USER1 diff --git a/onvm/onvm_nflib/onvm_flow_dir.c b/onvm/onvm_nflib/onvm_flow_dir.c index 444b1d0f6..f5eba6daf 100644 --- a/onvm/onvm_nflib/onvm_flow_dir.c +++ b/onvm/onvm_nflib/onvm_flow_dir.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm/onvm_nflib/onvm_flow_dir.h b/onvm/onvm_nflib/onvm_flow_dir.h index bfec8cbec..3e7deada9 100644 --- a/onvm/onvm_nflib/onvm_flow_dir.h +++ b/onvm/onvm_nflib/onvm_flow_dir.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm/onvm_nflib/onvm_flow_table.c b/onvm/onvm_nflib/onvm_flow_table.c index f54b568df..a95b0e01b 100644 --- a/onvm/onvm_nflib/onvm_flow_table.c +++ b/onvm/onvm_nflib/onvm_flow_table.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm/onvm_nflib/onvm_flow_table.h b/onvm/onvm_nflib/onvm_flow_table.h index d0c78e074..761c9f740 100644 --- a/onvm/onvm_nflib/onvm_flow_table.h +++ b/onvm/onvm_nflib/onvm_flow_table.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm/onvm_nflib/onvm_includes.h b/onvm/onvm_nflib/onvm_includes.h index 55d09c2b1..d2fc15638 100644 --- a/onvm/onvm_nflib/onvm_includes.h +++ b/onvm/onvm_nflib/onvm_includes.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. All rights reserved. * All rights reserved. * diff --git a/onvm/onvm_nflib/onvm_msg_common.h b/onvm/onvm_nflib/onvm_msg_common.h index 14352e443..f80ab84ef 100644 --- a/onvm/onvm_nflib/onvm_msg_common.h +++ b/onvm/onvm_nflib/onvm_msg_common.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation * All rights reserved. * diff --git a/onvm/onvm_nflib/onvm_nflib.c b/onvm/onvm_nflib/onvm_nflib.c index 18aa18a37..d90e8550a 100644 --- a/onvm/onvm_nflib/onvm_nflib.c +++ b/onvm/onvm_nflib/onvm_nflib.c @@ -5,9 +5,9 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside - * 2016 Hewlett Packard Enterprise Development LP + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -67,9 +67,6 @@ /**********************************Macros*************************************/ -// Number of packets to attempt to read from queue -#define PKT_READ_SIZE ((uint16_t)32) - // Possible NF packet consuming modes #define NF_MODE_UNKNOWN 0 #define NF_MODE_SINGLE 1 @@ -94,8 +91,15 @@ static struct rte_ring *nf_msg_ring; // rings used to pass packets between NFlib and NFmgr static struct rte_ring *tx_ring, *rx_ring; +//data for NFs that handle tx. May not be used +static struct queue_mgr *nf_tx_mgr; + // Shared data from server. We update statistics here -static struct onvm_nf *nfs; +struct onvm_nf *nfs; + +// Shared data from manager, has information used for nf_side tx +uint16_t **services; +uint16_t *nf_per_service_count; // Shared data for NF info extern struct onvm_nf_info *nf_info; @@ -119,7 +123,7 @@ static uint8_t keep_running = 1; static uint8_t nf_mode = NF_MODE_UNKNOWN; // Shared data for default service chain -static struct onvm_service_chain *default_chain; +struct onvm_service_chain *default_chain; /***********************Internal Functions Prototypes*************************/ @@ -135,6 +139,16 @@ static struct onvm_service_chain *default_chain; static struct onvm_nf_info * onvm_nflib_info_init(const char *tag); +/* + * Function that initialize a nf tx info data structure. + * + * Input : void + * Output : the data structure initialized + * + */ +static void +onvm_nflib_nf_tx_mgr_init(void); + /* * Function printing an explanation of command line instruction for a NF. @@ -170,7 +184,7 @@ onvm_nflib_handle_signal(int sig); /* * Check if there are packets in this NF's RX Queue and process them */ -static inline void +static inline uint16_t onvm_nflib_dequeue_packets(void **pkts, struct onvm_nf_info *info, pkt_handler handler) __attribute__((always_inline)); /* @@ -195,12 +209,14 @@ onvm_nflib_init(int argc, char *argv[], const char *nf_tag) { const struct rte_memzone *mz_nf; const struct rte_memzone *mz_port; const struct rte_memzone *mz_scp; + const struct rte_memzone *mz_services; + const struct rte_memzone *mz_nf_per_service; struct rte_mempool *mp; struct onvm_service_chain **scp; struct onvm_nf_msg *startup_msg; int retval_eal, retval_parse, retval_final; - if ((retval_eal = rte_eal_init(argc, argv)) < 0) + if ((retval_eal = rte_eal_init(argc, argv)) < 0) return -1; /* Modify argc and argv to conform to getopt rules for parse_nflib_args */ @@ -238,6 +254,9 @@ onvm_nflib_init(int argc, char *argv[], const char *nf_tag) { /* Initialize the info struct */ nf_info = onvm_nflib_info_init(nf_tag); + /* Initialize empty NF's tx manager */ + onvm_nflib_nf_tx_mgr_init(); + mp = rte_mempool_lookup(PKTMBUF_POOL_NAME); if (mp == NULL) rte_exit(EXIT_FAILURE, "Cannot get mempool for mbufs\n"); @@ -248,6 +267,18 @@ onvm_nflib_init(int argc, char *argv[], const char *nf_tag) { rte_exit(EXIT_FAILURE, "Cannot get NF structure mempool\n"); nfs = mz_nf->addr; + mz_services = rte_memzone_lookup(MZ_SERVICES_INFO); + if (mz_services == NULL) { + rte_exit(EXIT_FAILURE, "Cannot get service information\n"); + } + services = mz_services->addr; + + mz_nf_per_service = rte_memzone_lookup(MZ_NF_PER_SERVICE_INFO); + if (mz_nf_per_service == NULL) { + rte_exit(EXIT_FAILURE, "Cannot get NF per service information\n"); + } + nf_per_service_count = mz_nf_per_service->addr; + mz_port = rte_memzone_lookup(MZ_PORT_INFO); if (mz_port == NULL) rte_exit(EXIT_FAILURE, "Cannot get port info structure\n"); @@ -326,8 +357,9 @@ onvm_nflib_run_callback( pkt_handler handler, callback_handler callback) { - void *pkts[PKT_READ_SIZE]; + struct rte_mbuf *pkts[PACKET_READ_SIZE]; int ret; + uint16_t nb_pkts_added; /* Don't allow conflicting NF modes */ if (nf_mode == NF_MODE_RING) { @@ -347,7 +379,16 @@ onvm_nflib_run_callback( printf("[Press Ctrl-C to quit ...]\n"); for (; keep_running;) { - onvm_nflib_dequeue_packets(pkts, info, handler); + nb_pkts_added = onvm_nflib_dequeue_packets((void **) pkts, info, handler); + + if (likely(nb_pkts_added > 0)) { + onvm_pkt_process_tx_batch(nf_tx_mgr, pkts, nb_pkts_added, &nfs[info->instance_id]); + } + + /* Flush the packet buffers */ + onvm_pkt_enqueue_tx_thread(nf_tx_mgr->to_tx_buf, info->instance_id); + onvm_pkt_flush_all_nfs(nf_tx_mgr); + onvm_nflib_dequeue_messages(); if (callback != ONVM_NO_CALLBACK) { keep_running = !(*callback)() && keep_running; @@ -365,7 +406,6 @@ onvm_nflib_run(struct onvm_nf_info* info, pkt_handler handler) { return onvm_nflib_run_callback(info, handler, ONVM_NO_CALLBACK); } - int onvm_nflib_return_pkt(struct rte_mbuf* pkt) { /* FIXME: should we get a batch of buffered packets and then enqueue? Can we keep stats? */ @@ -456,45 +496,48 @@ onvm_nflib_get_nf(uint16_t id) { return &nfs[id]; } +struct onvm_service_chain * +onvm_nflib_get_default_chain(void) { + return default_chain; +} /******************************Helper functions*******************************/ -static inline void +static inline uint16_t onvm_nflib_dequeue_packets(void **pkts, struct onvm_nf_info *info, pkt_handler handler) { struct onvm_pkt_meta* meta; - uint16_t i, j, nb_pkts; - void *pktsTX[PKT_READ_SIZE]; - int tx_batch_size = 0; + uint16_t i, nb_pkts; + struct packet_buf tx_buf; int ret_act; /* Dequeue all packets in ring up to max possible. */ - nb_pkts = rte_ring_dequeue_burst(rx_ring, pkts, PKT_READ_SIZE); + nb_pkts = rte_ring_dequeue_burst(rx_ring, pkts, PACKET_READ_SIZE); + /* Probably want to comment this out */ if(unlikely(nb_pkts == 0)) { - return; + return 0; } + tx_buf.count = 0; + /* Give each packet to the user proccessing function */ for (i = 0; i < nb_pkts; i++) { meta = onvm_get_pkt_meta((struct rte_mbuf*)pkts[i]); ret_act = (*handler)((struct rte_mbuf*)pkts[i], meta); /* NF returns 0 to return packets or 1 to buffer */ if(likely(ret_act == 0)) { - pktsTX[tx_batch_size++] = pkts[i]; + tx_buf.buffer[tx_buf.count++] = pkts[i]; } else { nfs[info->instance_id].stats.tx_buffer++; } } + if (ONVM_NF_HANDLE_TX) { + return nb_pkts; + } - if (unlikely(tx_batch_size > 0 && rte_ring_enqueue_bulk(tx_ring, pktsTX, tx_batch_size) == -ENOBUFS)) { - nfs[info->instance_id].stats.tx_drop += tx_batch_size; - for (j = 0; j < tx_batch_size; j++) { - rte_pktmbuf_free(pktsTX[j]); - } - } else { - nfs[info->instance_id].stats.tx += tx_batch_size; - } + onvm_pkt_enqueue_tx_thread(&tx_buf, info->instance_id); + return 0; } static inline void @@ -534,6 +577,16 @@ onvm_nflib_info_init(const char *tag) return info; } +static void +onvm_nflib_nf_tx_mgr_init(void) +{ + nf_tx_mgr = calloc(1, sizeof(struct queue_mgr)); + nf_tx_mgr->mgr_type_t = NF; + nf_tx_mgr->to_tx_buf = calloc(1, sizeof(struct packet_buf)); + nf_tx_mgr->id = initial_instance_id; + nf_tx_mgr->nf_rx_bufs = calloc(MAX_NFS, sizeof(struct packet_buf)); +} + static void onvm_nflib_usage(const char *progname) { diff --git a/onvm/onvm_nflib/onvm_nflib.h b/onvm/onvm_nflib/onvm_nflib.h index 866b5903b..e62de6d74 100644 --- a/onvm/onvm_nflib/onvm_nflib.h +++ b/onvm/onvm_nflib/onvm_nflib.h @@ -5,9 +5,9 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside - * 2016 Hewlett Packard Enterprise Development LP + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2016-2017 Hewlett Packard Enterprise Development LP * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -54,6 +54,7 @@ #define _ONVM_NFLIB_H_ #include #include "onvm_common.h" +#include "onvm_pkt_common.h" /************************************API**************************************/ @@ -190,5 +191,7 @@ onvm_nflib_get_rx_ring(struct onvm_nf_info* info); struct onvm_nf * onvm_nflib_get_nf(uint16_t id); +struct onvm_service_chain * +onvm_nflib_get_default_chain(void); #endif // _ONVM_NFLIB_H_ diff --git a/onvm/onvm_nflib/onvm_pkt_common.c b/onvm/onvm_nflib/onvm_pkt_common.c new file mode 100644 index 000000000..38d5826e9 --- /dev/null +++ b/onvm/onvm_nflib/onvm_pkt_common.c @@ -0,0 +1,329 @@ +/********************************************************************* + * openNetVM + * https://sdnfv.github.io + * + * BSD LICENSE + * + * Copyright(c) + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + ********************************************************************/ + + +/****************************************************************************** + onvm_pkt_common.c + + This file contains all functions related to receiving or + transmitting packets. + +******************************************************************************/ + + +#include "onvm_pkt_common.h" + + +/**********************Internal Functions Prototypes**************************/ + + +/* + * Function to enqueue a packet on one port's queue. + * + * Inputs : a pointer to the tx queue responsible + * the number of the port + * a pointer to the packet + * + */ +static inline void +onvm_pkt_enqueue_port(struct queue_mgr *tx_mgr, uint16_t port, struct rte_mbuf *buf); + +/* + * Function to process a single packet. + * + * Inputs : a pointer to the tx queue responsible + * a pointer to the packet + * a pointer to the NF involved + * + */ +static inline void +onvm_pkt_process_next_action(struct queue_mgr *tx_mgr, struct rte_mbuf *pkt, struct onvm_nf *nf); + + +/* + * Helper function to drop a packet. + * + * Input : a pointer to the packet + * + * Ouput : an error code + * + */ +static int +onvm_pkt_drop(struct rte_mbuf *pkt); + + +/**********************************Interfaces*********************************/ + + +void +onvm_pkt_process_tx_batch(struct queue_mgr *tx_mgr, struct rte_mbuf *pkts[], uint16_t tx_count, struct onvm_nf *nf) { + uint16_t i; + struct onvm_pkt_meta *meta; + struct packet_buf *out_buf; + + if (tx_mgr == NULL || pkts == NULL || nf == NULL) + return; + + for (i = 0; i < tx_count; i++) { + meta = (struct onvm_pkt_meta*) &(((struct rte_mbuf*)pkts[i])->udata64); + meta->src = nf->instance_id; + if (meta->action == ONVM_NF_ACTION_DROP) { + // if the packet is drop, then is 0 + // and ! is 1. + nf->stats.act_drop += !onvm_pkt_drop(pkts[i]); + } else if (meta->action == ONVM_NF_ACTION_NEXT) { + /* TODO: Here we drop the packet : there will be a flow table + in the future to know what to do with the packet next */ + nf->stats.act_next++; + onvm_pkt_process_next_action(tx_mgr, pkts[i], nf); + } else if (meta->action == ONVM_NF_ACTION_TONF) { + nf->stats.act_tonf++; + onvm_pkt_enqueue_nf(tx_mgr, meta->destination, pkts[i]); + } else if (meta->action == ONVM_NF_ACTION_OUT) { + if (tx_mgr->mgr_type_t == MGR) { + nf->stats.act_out++; + onvm_pkt_enqueue_port(tx_mgr, meta->destination, pkts[i]); + } else { + out_buf = tx_mgr->to_tx_buf; + out_buf->buffer[out_buf->count++] = pkts[i]; + if (out_buf->count == PACKET_READ_SIZE) { + onvm_pkt_enqueue_tx_thread(out_buf, nf->instance_id); + } + } + } else { + printf("ERROR invalid action : this shouldn't happen.\n"); + onvm_pkt_drop(pkts[i]); + return; + } + } +} + +void +onvm_pkt_flush_all_nfs(struct queue_mgr *tx_mgr) { + uint16_t i; + + if (tx_mgr == NULL) + return; + + for (i = 0; i < MAX_NFS; i++) + onvm_pkt_flush_nf_queue(tx_mgr, i); +} + +void +onvm_pkt_flush_nf_queue(struct queue_mgr *tx_mgr, uint16_t nf_id) { + uint16_t i; + struct onvm_nf *nf; + struct packet_buf *nf_buf; + + if (tx_mgr == NULL) + return; + + nf_buf = &tx_mgr->nf_rx_bufs[nf_id]; + if (nf_buf->count == 0) + return; + + nf = &nfs[nf_id]; + + // Ensure destination NF is running and ready to receive packets + if (!onvm_nf_is_valid(nf)) + return; + + if (rte_ring_enqueue_bulk(nf->rx_q, (void **)nf_buf->buffer, + nf_buf->count) != 0) { + for (i = 0; i < nf_buf->count; i++) { + onvm_pkt_drop(nf_buf->buffer[i]); + } + nf->stats.rx_drop += nf_buf->count; + } else { + nf->stats.rx += nf_buf->count; + } + nf_buf->count = 0; +} + +inline void +onvm_pkt_enqueue_nf(struct queue_mgr *tx_mgr, uint16_t dst_service_id, struct rte_mbuf *pkt) { + struct onvm_nf *nf; + uint16_t dst_instance_id; + struct packet_buf *nf_buf; + + + if (tx_mgr == NULL || pkt == NULL) + return; + + // map service to instance and check one exists + dst_instance_id = onvm_sc_service_to_nf_map(dst_service_id, pkt); + if (dst_instance_id == 0) { + onvm_pkt_drop(pkt); + return; + } + + // Ensure destination NF is running and ready to receive packets + nf = &nfs[dst_instance_id]; + if (!onvm_nf_is_valid(nf)) { + onvm_pkt_drop(pkt); + return; + } + + nf_buf = &tx_mgr->nf_rx_bufs[dst_instance_id]; + nf_buf->buffer[nf_buf->count++] = pkt; + if (nf_buf->count == PACKET_READ_SIZE) { + onvm_pkt_flush_nf_queue(tx_mgr, dst_instance_id); + } +} + +void +onvm_pkt_flush_port_queue(struct queue_mgr *tx_mgr, uint16_t port) { + uint16_t i, sent; + volatile struct tx_stats *tx_stats; + struct packet_buf *port_buf; + + if (tx_mgr == NULL || tx_mgr->mgr_type_t != MGR) + return; + + port_buf = &tx_mgr->tx_thread_info->port_tx_bufs[port]; + if (port_buf->count == 0) + return; + + tx_stats = &(ports->tx_stats); + sent = rte_eth_tx_burst(port, + tx_mgr->id, + port_buf->buffer, + port_buf->count); + if (unlikely(sent < port_buf->count)) { + for (i = sent; i < port_buf->count; i++) { + onvm_pkt_drop(port_buf->buffer[i]); + } + tx_stats->tx_drop[port] += (port_buf->count - sent); + } + tx_stats->tx[port] += sent; + + port_buf->count = 0; +} + +void +onvm_pkt_enqueue_tx_thread(struct packet_buf *pkt_buf, uint16_t instance_id) { + uint16_t i; + + if (pkt_buf->count == 0) + return; + + if (unlikely(pkt_buf->count > 0 && rte_ring_enqueue_bulk(nfs[instance_id].tx_q, (void **)pkt_buf->buffer, pkt_buf->count) == -ENOBUFS)) { + nfs[instance_id].stats.tx_drop += pkt_buf->count; + for (i = 0; i < pkt_buf->count; i++) { + rte_pktmbuf_free(pkt_buf->buffer[i]); + } + } else { + nfs[instance_id].stats.tx += pkt_buf->count; + } + pkt_buf->count = 0; +} + + +/****************************Internal functions*******************************/ + + +inline static void +onvm_pkt_enqueue_port(struct queue_mgr *tx_mgr, uint16_t port, struct rte_mbuf *buf) { + struct packet_buf *port_buf; + + if (tx_mgr == NULL || buf == NULL) + return; + + port_buf = &tx_mgr->tx_thread_info->port_tx_bufs[port]; + port_buf->buffer[port_buf->count++] = buf; + if (port_buf->count == PACKET_READ_SIZE) { + onvm_pkt_flush_port_queue(tx_mgr, port); + } +} + + +inline static void +onvm_pkt_process_next_action(struct queue_mgr *tx_mgr, struct rte_mbuf *pkt, struct onvm_nf *nf) { + + if (tx_mgr == NULL || pkt == NULL || nf == NULL) + return; + + struct onvm_flow_entry *flow_entry; + struct onvm_service_chain *sc; + struct onvm_pkt_meta *meta = onvm_get_pkt_meta(pkt); + int ret; + + ret = onvm_flow_dir_get_pkt(pkt, &flow_entry); + if (ret >= 0) { + sc = flow_entry->sc; + meta->action = onvm_sc_next_action(sc, pkt); + meta->destination = onvm_sc_next_destination(sc, pkt); + } else { + meta->action = onvm_sc_next_action(default_chain, pkt); + meta->destination = onvm_sc_next_destination(default_chain, pkt); + } + + switch (meta->action) { + case ONVM_NF_ACTION_DROP: + // if the packet is drop, then is 0 + // and ! is 1. + nf->stats.act_drop += !onvm_pkt_drop(pkt); + break; + case ONVM_NF_ACTION_TONF: + nf->stats.act_tonf++; + onvm_pkt_enqueue_nf(tx_mgr, meta->destination, pkt); + break; + case ONVM_NF_ACTION_OUT: + nf->stats.act_out++; + onvm_pkt_enqueue_port(tx_mgr, meta->destination, pkt); + break; + default: + break; + } + (meta->chain_index)++; +} + + +/*******************************Helper function*******************************/ + + +static int +onvm_pkt_drop(struct rte_mbuf *pkt) { + rte_pktmbuf_free(pkt); + if (pkt != NULL) { + return 1; + } + return 0; +} diff --git a/onvm/onvm_nflib/onvm_pkt_common.h b/onvm/onvm_nflib/onvm_pkt_common.h new file mode 100644 index 000000000..a4b39ce15 --- /dev/null +++ b/onvm/onvm_nflib/onvm_pkt_common.h @@ -0,0 +1,126 @@ +/********************************************************************* + * openNetVM + * https://sdnfv.github.io + * + * BSD LICENSE + * + * Copyright(c) + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside + * 2010-2014 Intel Corporation. All rights reserved. + * All rights reserved. + * + * Redistribution and use in source and binary forms, with or without + * modification, are permitted provided that the following conditions + * are met: + * + * * Redistributions of source code must retain the above copyright + * notice, this list of conditions and the following disclaimer. + * * Redistributions in binary form must reproduce the above copyright + * notice, this list of conditions and the following disclaimer in + * the documentation and/or other materials provided with the + * distribution. + * * The name of the author may not be used to endorse or promote + * products derived from this software without specific prior + * written permission. + * + * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS + * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT + * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR + * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT + * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, + * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT + * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, + * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY + * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT + * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE + * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. + * + ********************************************************************/ + + +/****************************************************************************** + + onvm_pkt_common.h + + + Header file containing all prototypes of packet processing functions + + +******************************************************************************/ + + +#ifndef _ONVM_PKT_COMMON_H_ +#define _ONVM_PKT_COMMON_H_ + +#include "onvm_common.h" +#include "onvm_sc_common.h" +#include "onvm_includes.h" +#include "onvm_flow_dir.h" +#include "onvm_sc_mgr.h" + +extern struct port_info *ports; +extern struct onvm_service_chain *default_chain; + +/*********************************Interfaces**********************************/ + + +/* + * Interface to process packets in a given TX queue. + * + * Inputs : a pointer to the tx queue + * an array of packets + * the size of the array + * a pointer to the NF possessing the TX queue. + * + */ +void +onvm_pkt_process_tx_batch(struct queue_mgr *tx_mgr, struct rte_mbuf *pkts[], uint16_t tx_count, struct onvm_nf *nf); + +/* + * Interface to send packets to all NFs after processing them. + * + * Input : a pointer to the tx queue + * + */ +void +onvm_pkt_flush_all_nfs(struct queue_mgr *tx_mgr); + +/* + * Function to send packets to one NF after processing them. + * + * Input : a pointer to the tx queue + * + */ + +void +onvm_pkt_flush_nf_queue(struct queue_mgr *tx_mgr, uint16_t nf_id); + +/* + * Function to enqueue a packet on one NF's queue. + * + * Inputs : a pointer to the tx queue responsible + * the number of the port + * a pointer to the packet + * + */ +inline void +onvm_pkt_enqueue_nf(struct queue_mgr *tx_mgr, uint16_t dst_service_id, struct rte_mbuf *pkt); + +/* + * Function to send packets to one port after processing them. + * + * Input : a pointer to the tx queue + * + */ +void +onvm_pkt_flush_port_queue(struct queue_mgr *tx_mgr, uint16_t port); + +/* + * Give packets to TX thread so it can do useful work. + */ +void +onvm_pkt_enqueue_tx_thread(struct packet_buf *pkt_buf, uint16_t instance_id); + + +#endif // _ONVM_PKT_COMMON_H_ diff --git a/onvm/onvm_nflib/onvm_pkt_helper.c b/onvm/onvm_nflib/onvm_pkt_helper.c index a73114429..05bab0924 100644 --- a/onvm/onvm_nflib/onvm_pkt_helper.c +++ b/onvm/onvm_nflib/onvm_pkt_helper.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm/onvm_nflib/onvm_pkt_helper.h b/onvm/onvm_nflib/onvm_pkt_helper.h index 25691a0ea..472e0b7ea 100644 --- a/onvm/onvm_nflib/onvm_pkt_helper.h +++ b/onvm/onvm_nflib/onvm_pkt_helper.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm/onvm_nflib/onvm_sc_common.c b/onvm/onvm_nflib/onvm_sc_common.c index eef58a04d..cb8af8917 100644 --- a/onvm/onvm_nflib/onvm_sc_common.c +++ b/onvm/onvm_nflib/onvm_sc_common.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -43,6 +43,26 @@ #include "onvm_common.h" #include "onvm_sc_common.h" +/*********************************Interfaces**********************************/ + +uint16_t +onvm_sc_service_to_nf_map(uint16_t service_id, struct rte_mbuf *pkt) { + if (!nf_per_service_count || !services) { + rte_exit(EXIT_FAILURE, "Failed to retrieve service information\n"); + } + uint16_t num_nfs_available = nf_per_service_count[service_id]; + + if (num_nfs_available == 0) + return 0; + + if (pkt == NULL) + return 0; + + uint16_t instance_index = pkt->hash.rss % num_nfs_available; + uint16_t instance_id = services[service_id][instance_index]; + return instance_id; +} + int onvm_sc_append_entry(struct onvm_service_chain *chain, uint8_t action, uint16_t destination) { int chain_length = chain->chain_length; @@ -67,7 +87,6 @@ onvm_sc_set_entry(struct onvm_service_chain *chain, int entry, uint8_t action, u chain->sc[entry].action = action; chain->sc[entry].destination = destination; - return 0; } diff --git a/onvm/onvm_nflib/onvm_sc_common.h b/onvm/onvm_nflib/onvm_sc_common.h index c7342ef90..1777c4332 100644 --- a/onvm/onvm_nflib/onvm_sc_common.h +++ b/onvm/onvm_nflib/onvm_sc_common.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without @@ -44,6 +44,17 @@ #include #include "onvm_common.h" +/********************************Global variables*****************************/ + +extern struct onvm_nf *nfs; +extern uint16_t **services; +extern uint16_t *nf_per_service_count; + +/********************************Interfaces***********************************/ + +uint16_t +onvm_sc_service_to_nf_map(uint16_t service_id, struct rte_mbuf *pkt);/*, uint16_t *nf_per_service_count, uint16_t **services);*/ + /* append a entry to serivce chain, 0 means appending successful, 1 means failed*/ int onvm_sc_append_entry(struct onvm_service_chain *chain, uint8_t action, uint16_t destination); @@ -51,4 +62,5 @@ int onvm_sc_append_entry(struct onvm_service_chain *chain, uint8_t action, uint1 int onvm_sc_set_entry(struct onvm_service_chain *chain, int entry, uint8_t action, uint16_t destination); void onvm_sc_print(struct onvm_service_chain *chain); + #endif //_SC_COMMON_H_ diff --git a/onvm/onvm_nflib/onvm_sc_mgr.c b/onvm/onvm_nflib/onvm_sc_mgr.c index 43c4b2844..527848961 100644 --- a/onvm/onvm_nflib/onvm_sc_mgr.c +++ b/onvm/onvm_nflib/onvm_sc_mgr.c @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm/onvm_nflib/onvm_sc_mgr.h b/onvm/onvm_nflib/onvm_sc_mgr.h index b28b1faec..f3e73c170 100644 --- a/onvm/onvm_nflib/onvm_sc_mgr.h +++ b/onvm/onvm_nflib/onvm_sc_mgr.h @@ -5,8 +5,8 @@ * BSD LICENSE * * Copyright(c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * All rights reserved. * * Redistribution and use in source and binary forms, with or without diff --git a/onvm_web/start_web_console.sh b/onvm_web/start_web_console.sh index 4749a540f..ebf1883e0 100755 --- a/onvm_web/start_web_console.sh +++ b/onvm_web/start_web_console.sh @@ -6,8 +6,8 @@ # OpenNetVM is distributed under the following BSD LICENSE: # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/scripts/README.md b/scripts/README.md index 90e33bd6c..23646500e 100644 --- a/scripts/README.md +++ b/scripts/README.md @@ -9,8 +9,8 @@ portions of openNetVM # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # 2010-2014 Intel Corporation. # All rights reserved. # diff --git a/scripts/corehelper.py b/scripts/corehelper.py index 88a31a026..1709b2a18 100755 --- a/scripts/corehelper.py +++ b/scripts/corehelper.py @@ -6,8 +6,8 @@ # BSD LICENSE # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # 2010-2014 Intel Corporation. # All rights reserved. # diff --git a/scripts/docker.sh b/scripts/docker.sh index 320f11079..144c7eea9 100755 --- a/scripts/docker.sh +++ b/scripts/docker.sh @@ -6,8 +6,8 @@ # OpenNetVM is distributed under the following BSD LICENSE: # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/scripts/install.sh b/scripts/install.sh index a5f81e619..937c1e2e2 100755 --- a/scripts/install.sh +++ b/scripts/install.sh @@ -6,8 +6,8 @@ # OpenNetVM is distributed under the following BSD LICENSE: # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/scripts/setup_cloudlab.sh b/scripts/setup_cloudlab.sh index 2198eb116..52b14a7ab 100755 --- a/scripts/setup_cloudlab.sh +++ b/scripts/setup_cloudlab.sh @@ -6,8 +6,8 @@ # OpenNetVM is distributed under the following BSD LICENSE: # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/scripts/setup_environment.sh b/scripts/setup_environment.sh index e2865ea0f..d8b3aadc1 100755 --- a/scripts/setup_environment.sh +++ b/scripts/setup_environment.sh @@ -6,8 +6,8 @@ # OpenNetVM is distributed under the following BSD LICENSE: # # Copyright(c) -# 2015-2016 George Washington University -# 2015-2016 University of California Riverside +# 2015-2017 George Washington University +# 2015-2017 University of California Riverside # All rights reserved. # # Redistribution and use in source and binary forms, with or without diff --git a/style/gwclint.py b/style/gwclint.py index 8ad40db3b..9d24e7303 100644 --- a/style/gwclint.py +++ b/style/gwclint.py @@ -1,7 +1,7 @@ #!/usr/bin/env python # # Copyright (c) 2009 Google Inc. All rights reserved. -# Copyright (c) 2015-2016 George Washington University - modifications for C support +# Copyright (c) 2015-2017 George Washington University - modifications for C support # # Redistribution and use in source and binary forms, with or without # modification, are permitted provided that the following conditions are diff --git a/style/styleguide.md b/style/styleguide.md index fbe12388f..3a1c12354 100644 --- a/style/styleguide.md +++ b/style/styleguide.md @@ -1,6 +1,6 @@ # CloudLab Coding Style Guide -*Updated June 21, 2016* +*Updated June 21, 2017* As a student, you have spent years writing code that was read by at most two or three people. This has allowed you to get away with terrible coding habits. In the real world, writing good code is **important**, but writing readable code is **critical**. @@ -32,8 +32,8 @@ Your project is likely to include multiple files, so you need to think about how * BSD LICENSE * * Copyright (c) - * 2015-2016 George Washington University - * 2015-2016 University of California Riverside + * 2015-2017 George Washington University + * 2015-2017 University of California Riverside * 2010-2014 Intel Corporation. * All rights reserved. *