Follow @Openwall on Twitter for new release announcements and other news
[<prev] [next>] [thread-next>] [day] [month] [year] [list]
Date: Mon, 29 Jun 2020 10:12:50 +0800 (CST)
From: youyan  <hyouyan@....com>
To: lkrg-users@...ts.openwall.com
Subject: p_install_arch_jump_label_transform_hook and p_check_integrity lead
 to deadlock issue on unisoc SL8541E

Hi adam

   I have found the reason,why LKRG block on my device.

   1: hardware and software: unisoc SL8541E, android Q, kernel version is 4.14

   2: SL8541E is arm64 platform, but for some reason, we compile it as 32bit arm

   3: On function p_create_database(),which in the file p_database.c,have the fellow execute flow:

      (1) if (p_register_arch_metadata() != P_LKRG_SUCCESS) 
[  125.693391] c1 [<c01e379c>] (kick_kprobe_optimizer) from [<c01e4394>] (optimize_kprobe+0x108/0x118)
[  125.702389] c1 [<c01e4394>] (optimize_kprobe) from [<c01e5fe0>] (register_kprobe+0x548/0x5b0)
[  125.710871] c1 [<c01e5fe0>] (register_kprobe) from [<c01e63ac>] (register_kretprobe+0x114/0x178)
[  125.719671] c1 [<c01e63ac>] (register_kretprobe) from [<bf2d9f60>] (p_install_arch_jump_label_transform_hook+0x38/0xc0 [p_lkrg])
[  125.731224] c1 [<bf2d9f60>] (p_install_arch_jump_label_transform_hook [p_lkrg]) from [<bf2d9b98>] (p_register_arch_metadata+0x74/0xd0 [p_lkrg])
     (2) kick_kprobe_optimizer()->schedule_delayed_work(&optimizing_work, OPTIMIZE_DELAY)->kprobe_optimizer()->mutex_lock(&module_mutex)->do_optimize_kprobes()->mutex_lock(&text_mutex)

     (3) p_register_module_notifier();

   4: some notifier or timer can trigger p_check_integrity()

   5: p_module_event_notifier is execute after the p_register_module_notifier:

   p_module_event_notifier_live_retry:

         p_text_section_lock();

         /* We are heavily consuming module list here - take 'module_mutex' */

         //mutex_lock(&module_mutex);

         while (!mutex_trylock(&module_mutex)) {

            p_text_section_unlock();

            goto  p_module_event_notifier_live_retry;

         }  

   above will loop request exceute,and will consume a lot of cpu resource. It lead to do_optimize_kprobes can't get mutex_lock(&text_mutex);

   7:when p_check_integrity execute,it first execute  p_text_section_lock(),then mutex_lock(&module_mutex), and lead to deadlock.

    

   kprobe.c                                                   p_integrity_timer.c

   mutex_lock(&module_mutex)              

                                                                   p_text_section_lock()->mutex_lock(P_SYM(p_text_mutex));

   

                                                                   mutex_lock(&text_mutex)

   mutex_lock(&text_mutex)

   

   8:Maybe SL8541E run slowly cause this bug.

   9:I try fellow three solutions to fix this issue:

      solution one:

        Before p_register_module_notifier(),add a some delay use msleep();

      solution two:

          (1) p_module_event_notifier request mutex_trylock(&module_mutex) loop,add msleep(10):

        p_module_event_notifier_going_retry:

      p_text_section_lock();

      while (!mutex_trylock(&module_mutex)) {

  msleep(10);

         p_text_section_unlock();

         goto  p_module_event_notifier_going_retry;

      }

  (2)change request mutex order

      p_text_section_lock();                                 mutex_lock(&module_mutex);

                                     change to           

      mutex_lock(&module_mutex);                     p_text_section_lock();

      solution third:

p_text_section_lock(); p_check_integrity_mutex:

                                                                                            change to

       mutex_lock(&module_mutex);                p_text_section_lock();

                                                     while (!mutex_trylock(&module_mutex)) {

                                             p_text_section_unlock();

 msleep(10);

goto  p_check_integrity_mutex;

                         }

    10:I maybe use use solution one, because I am not familiar with LKRG. Which solution do you suggest? If you have better idea,could you share me? Thanks!!!





 
Content of type "text/html" skipped

Powered by blists - more mailing lists

Confused about mailing lists and their use? Read about mailing lists on Wikipedia and check out these guidelines on proper formatting of your messages.