2016년 8월 1일 월요일

PJSUA Function Flow-Media

PJSUA Function Flow-Media


기본 미디어 포트 등록

app_init() : pjsip-apps/src/pjsua/pjsua_app.c
…
재생 및 녹화 파일 등록-
- pjsua_player_create() -> pjmedia_wav_player_port_create(), pjsua_conf_add_port()
- pjsua_recorder_create() -> pjmedia_wav_writer_port_create(), pjsua_conf_add_port()
미디어포트 생성
- pjmedia_tonegen_create2(..., “tone-freq1,freq2”, …), pjsua_conf_add_port(..., tport, ...)
- pjmedia_tonegen_create2(..., “ringback”, …), pjsua_conf_add_port(..., ringback_port, ...)
- pjmedia_tonegen_create2(..., “ring”, …), pjsua_conf_add_port(..., ring_port, …)
전송포트 생성
- pjsua_transport_create(PJSIP_TRANSPORT_UDP, ...)
- pjsua_transport_create(PJSIP_TRANSPORT_TCP, ...)
- pjsua_transport_create(PJSIP_TRANSPORT_TLS, …)

오디오장치 설정 및 스트림 시작흐름

전화발신시
*Make outgoing call to the specified URI using the specified account.
pjsip/src/pjsua-lib/pjsua_call.c :: pjsua_call_make_call()
 * Create outgoing dialog:
 ->pjsip_dlg_create_uac()
 * Outgoing call callback when media transport creation is completed
 -> pjsua_call.c  :: on_make_call_med_tp_complete()
   * Create UAC invite session
   -> pjsip/src/pjsip-ua/sip_inv.c :: pjsip_inv_create_uac()  ->  pjsip_dlg_add_usage()
* Initialize invite session callback
pjsip/src/pjsua-lib/pjsua_call.c:
...
inv_cb.on_media_update = &pjsua_call_on_media_update;
...
* Initialize invite session module.
pjsip_inv_usage_init(pjsua_var.endpt, &inv_cb)
inv_check_sdp_in_incoming_msg() : Check in incoming message for SDP offer/answer.
->> inv_negotiate_sdp() : Initiate SDP negotiation in the SDP negotiator.
->> mod_inv.cb.on_media_update)(inv, status)

전화수신시

mod-pjsua.on_rx_request()-> app_config.cfg.cb.on_incoming_call()->
pjsip_inv_answer()->inv_negotiate_sdp()->inv_cb.on_media_update() ->  app_config.cfg.cb.on_call_media_state()
-> on_call_audio_state() ->pjsua_conf_cnnect()
-> pjsua_set_snd_dev()
 ->> pjmedia_snd_port_create2()

PJSUA에서 호출되지 않는 포트생성 함수

컨퍼런트 포트 생성시

pjmedia_snd_port_create()
  • pjmedia_conf_create()에서 pjmedia_conf.options에 PJMEDIA_CONF_NO_DEVICE 이 없으면 수행된다. 하지만 pjsua는 PJMEDIA_CONF_NO_DEVICE이 기본으로 설정되어 호출되지 않는다.
 -> param.base.dir = PJMEDIA_DIR_CAPTURE_PLAYBACK
 ->  param.base.rec_id = dev_id;
  -> param.base.play_id = dev_id;
 -> pjmedia_snd_port_create2() <<<<<

레코딩만 하는 경우

pjmedia_snd_port_create_rec()
:pjmedia/src/pjmedia/sound_port.c
 :캡춰만 할경우 호출된다.
 -> param.base.dir = PJMEDIA_DIR_CAPTURE;
 -> param.base.rec_id = dev_id;
 -> pjmedia_snd_port_create2(param, p_port) <<<<<

플레이만 하는 경우

pjmedia_snd_port_create_player()
 : 플레이만 할 경우 호출된다. 예를 들면 ring, ringback, play-file의 경우.
 -> param.base.dir = PJMEDIA_DIR_PLAYBACK;
 -> param.base.play_id = dev_id;
 -> pjmedia_snd_port_create2() <<<<<

오디오장치 설정 및 스레드 시작

pjmedia_snd_port_create2(pool, prm, p_port)
 -> snd_port->rec_id = prm->base.rec_id 등 prm을 snd_port에 채움.
 -> pjmedia_clock_src_init(snd_port->cap_clocksrc,snd_port->clock_rate..)
 -> pjmedia_clock_src_init(snd_port->play_clocksrc,snd_port->clock_rate..)
 -> start_sound_device(pool, snd_port)
   ->> pjmedia_aud_stream_create(pjmedia_aud_param, rec_cb, play_cb...)  :pjmedia/src/pjmedia/audiodev.c
         - 오디오장치 오픈 및 placy_cb, rec_cb 콜백등록
        ->>> f->op->create_stream(f, &param, rec_cb, play_c...)
   ->> pjmedia_snd_port_set_ec() - AEC 설정
   ->> pjmedia_aud_stream_start(snd_port->aud_stream) :pjmedia/src/pjmedia/sound_port.c   
         - record/capture 스레드 시작
pjmedia_aud_dev_factory_op{
 ....
 create_stream = alsa_factory_create_stream()
}


alsa_factory_create_stream() :pjmedia/src/pjmedia-audiodev/alsa_dev.c
 -> stream->pb_cb     = play_cb;
    stream->ca_cb     = rec_cb;
    stream->base.op = &alsa_stream_op;
 -> open_playback (stream, param)
   ->> snd_pcm_open()
   ->> snd_pcm_hw_params_set_period_size_near()
   ->> snd_pcm_hw_params()
 -> open_capture (stream, param)
   ->> snd_pcm_open()
   ->> snd_pcm_hw_params_set_period_size_near()
   ->> snd_pcm_hw_params()
static pjmedia_aud_stream_op alsa_stream_op = :src/pjmedia-audiodev/alsa_dev.c
{
 ...
 set_cap = alsa_stream_get_cap
 start = alsa_stream_start,
 stop = alsa_stream_stop,
}

버퍼제어

alsa_stream_start(pjmedia_aud_stream *s) : pjmedia/src/pjmedia-audiodev/alsa_dev.c
 :struct alsa_stream *stream = s;
 -> pj_thread_create("alsasound_playback", pb_thread_func,stream...)
    ->> stream->pb_cb (user_data, &frame)
    ->> snd_pcm_writei (pcm, buf, nframes) -EPIPE 실패시::q  "pb_thread_func: underrun!" :확인해볼것!
 -> pj_thread_create("alsasound_capture", ca_thread_func,stream...)
    ->> snd_pcm_readi (pcm, buf, nframes)
    ->> stream->ca_cb (user_data, &frame)  -EPIPE 실패시: "ca_thread_func: overrun!"


rec_cb(void *user_data, frame) : pjmedia/sound_port.c
 : sound recoder 가 캡춰를 수행 후 이 함수를 호출한다.
 -> pjmedia_clock_src_update(snd_port->cap_clocksrc, frame->timestamp)
 -> pjmedia_ehco_cpature(frame->buf)
 -> pjmedia_port_put_frame(snd_port->port, frame) :pjmedia/port.c
play_cb(void *user_data, frame) : pjmedia/sound_port.c
 : sound player thread가 재생할 샘플이 필요할때 이 함수를 호출한다.  
 -> pjmedia_port_get_frame(snd_port->port, frame);
 -> pjmedia_clock_src_update(snd_port->play_clocksrc, frame->timestamp         
 -> pjmedia_echo_playback(fram->buf)


네트워크 스트림 포트생성

PJSUA log -------------------------------------------------------------
00:14:15.366    pjsua_aud.c  ..Conf connect: 3 --> 0
00:14:15.367    pjsua_aud.c  ...Set sound device: capture=-1, playback=-2
00:14:15.367    pjsua_app.c  ....Turning sound device ON
00:14:15.367    pjsua_aud.c  ....Opening sound device (speaker + mic) PCM@16000/1/20ms
00:14:15.442   ec0xb5a13ce0  .....AEC created, clock_rate=16000, channel=1, samples per frame=320, tail length=200 ms, latency=0 ms
00:14:15.443   conference.c  ...Port 3 (ring) transmitting to port 0 (default:CARD=SAMA5D3EK)
00:14:15.443   pjsua_call.c  ..Answering call 0: code=200
00:14:15.445  pjsua_media.c  .....Call 0: updating media..
00:14:15.447    pjsua_aud.c  ......Audio channel update..     <<<<<
---------------------------------------------------------------------------------------
전화수신시 함수 호출순서:
mod-pjsua.on_rx_request()-> app_config.cfg.cb.on_incoming_call()->
pjsip_inv_answer()->inv_negotiate_sdp()->inv_cb.on_media_update() ->  app_config.cfg.cb.on_call_media_state()
-> pjsua_aud_channel_update()
pjsua_aud_channel_update() :pjsip/src/pjsua-lib/pjsua_aud.c  <<<<<<
 -> pjmedia_stream_create() :/pjmedia/src/pjmedia/stream.c
   ->> stream = PJ_POOL_ZALLOC_T(pool, pjmedia_stream): 생성하고 이 함수에서 값을 채운다.
   ->> pjmedia_port_info_init(stream->port, clock_rate, channel_cnt, bitpersample..)
   ->> pjmedia_codec_init(stream->codec)
   ->> stream->port.put_frame = &put_frame
   ->> stream->port.get_frame = &get_frame_ext;
   ->> pjmedia_jbuf_create()
   ->> create_channel(..,PJMEDIA_DIR_DECODING, stream->dec)
   ->> create_channel(..,PJMEDIA_DIR_ENCODING, stream->enc)
   ->> pjmedia_rtcp_init2()
 -> pjmedia_stream_start(call_med->strm.a.stream)
    : stream->enc->paused = 0;
 -> pjmedia_stream_get_port(call_med->strm.a.stream, &media_port)
    : media_port 인터페이스 전달.(stream->port)
 -> pjmedia_conf_add_port(media_port, conf_slot) :pjmedia/src/pjmedia/conference.c
    : conference bride에 전화스트림 연결.
   ->> create_conf_port(pool, conf, strm_port, port_name, &conf_port)
     ->>> pjmedia_resample_create() for Rx Buffer
     ->>> pjmedia_resample_create() for Tx Buffer
     ->>> conf_port->rx_buf = alloc()
     ->>> conf_port->rx_buf = alloc()
     ->>> conf_port->mix_buf = alloc()

버퍼제어

   ->> port->put_frame(port, frame) :pjmedia/stream.c
        : pcm을 인코딩하고 RTP 패킷으로 만든다.(upstream)
        ->>> rebuffer(stream, frame) : frame-buf를 steam->enc_buf로 이동
        ->>> put_frame_imp(port, frame)
            ->>>> pjmedia_codec_encode(stream->codec, frame_in, size, frame_out);
            ->>>> pjmedia_rtp_encode_rtp(channel->rtp, channel->pt,...)
                 : frame_out = channel->out_pkt
 : fram_in = frame
 -> pjmedia_port_get_frame(port, frame) :pjmedia/port.c
   ->> port->get_frame(port, frame)
      : jitter buffer에서 정상적인 프레임을 가져와 디코딩한다.
      ->>> pjmedia_jbuf_get_frame2(stream->jb, channel->out_pkt, frame_size,....)       
      ->>> pjmedia_codec_decode(stream->codec, frame_in, size, frame_out)
           :frame_in.buf = channel->out_pkt
           :frame_out.buf = frame_out.buf = frame->buf + samples_count;

Coference Bridge

전화수신시 또는 파일 재생 연결시 포트생성:
PJSUA log -------------------------------------------------------------------------------------
00:14:10.096 sip_endpoint.c  Module "mod-default-handler" registered
00:14:10.096    pjsua_aud.c  Creating file player: /mnt/TheCubanMissileCrisis_8K.wav..
00:14:10.113   wav_player.c  .File player '/mnt/TheCubanMissileCrisis_8K.wav' created: samp.rate=8000, ch=1, bufsize=4KB, filesize=7264KB
00:14:10.113    pjsua_aud.c  .Player created, id=0, slot=1
.....
>>> cc 1 0
00:29:53.854    pjsua_aud.c !Conf connect: 1 --> 0
00:29:53.854    pjsua_aud.c  .Set sound device: capture=-1, playback=-2
00:29:53.854    pjsua_app.c  ..Turning sound device ON
00:29:53.854    pjsua_aud.c  ..Opening sound device (speaker + mic) PCM@16000/1/20ms
00:29:53.930     ec0x1bf8c8  ...AEC created, clock_rate=16000, channel=1, samples per frame=320, tail length=200 ms, latency=0 ms
00:29:53.930   conference.c  .Port 1 (/mnt/TheCubanMissileCrisis_8K.wav) transmitting to port 0 (default:CARD=SAMA5D3EK)
-----------------------------------------------------------------------------------------------------
pjsua_init()
pjsua_media_subsys_init() :src/pjsua-lib/pjsua_media.c
 -> pjsua_aud_subsys_init() :pjsua_aud.c
   ->> pjmedia_conf_create() :pjmedia/src/pjmedia/conference.c
     ->>> conf->master_port->get_frame = get_frame;
     ->>> conf->master_port->put_frame = &put_frame;
     ->>> create_sound_port() -  name:"Master/sound"
     **** device 의 play_cb, rec_cb에서 master_port의 get_frame, put_frame을 호출한다.
conf->master_port->put_frame()
-> conference :: put_frame()
 -> read_port() : :pjmedia/src/pjmedia/conference.c
   :Read from port.
   ->> pjmedia_port_get_frame(cport->port, frame) :pjmedia/port.c
   
conf->master_port->get_frame()
-> conference :: get_frame(frame)  : stream port에서 가져운 frame
 수신 대기중인 포트에 가져온 프레임을 전달한다.
 -> 등록된 모든포트(conf->ports[i])에서 수신대기포트의 mix_buff에 signal을 섞는다.
 -> write_port(conf, conf->ports[]i]) x conf->port_cnt :pjmedia/src/pjmedia/conference.c
   : 믹스된 신호를 port로 전달한다.
   : 믹스된 신호는 32bit다. mix buffer에 샘플이 있으면 32비트->16비트로 변경한다.
   : 레벨 변경이 필요하다면 tx signal에 적용한다.
   ->> frame.buf = cport->tx_buf
   ->> pjmedia_portput_frame(cport->port, frame) :pjmedia/port.c
       : coference에 등록된 포트에 쓴다.
 -> pjmedia_copy_samples(frame->buf, conf->ports[0]->mix_buf,conf->samples_per_frame)

작성자: 시간:
라벨: , , ,

댓글 없음:

댓글 쓰기

피드 구독하기: 댓글 (Atom)